Windows 7 Forums

Welcome to Windows 7 Forums. Our forum is dedicated to helping you find support and solutions for any problems regarding your Windows 7 PC be it Dell, HP, Acer, Asus or a custom build. We also provide an extensive Windows 7 tutorial section that covers a wide range of tips and tricks.


Windows 7: Another diskpart clean tale of woe

16 Jul 2016   #31
townsbg

Windows 7 pro 64-bit
 
 

Quote   Quote: Originally Posted by CurlyBen View Post
As for file systems, I understand that different developers have differing requirements and objectives, but given that OS X can read an NTFS volume without issue it really shouldn't be a big deal to implement basic write capability. There are drivers available to allow OSX to write to an NTFS partition but they seem to get mixed reviews, because they're not properly integrated into the system. I never tried reading HFS with a Windows machine, but it shouldn't be all that complicated. It's just quite frustrating as someone who uses both systems!
Quite a bold statement to make if you don't understand OS level programming. Just food for thought. I found this on Wikipedia. Apparently enabling write support on a mac causes it to crash so perhaps Apple tried and ran into problems they couldn't fix so they avoided them altogether by disabling it (not surprising considering Apple's history). I don't think that either company is in a hurry to make their systems inter-operable considering how much money that they are already making. If you think about it you can't (without an illegal hack) put mac on a pc and most that buy a mac aren't going to put windows on it so there isn't much of a demand for them to do so.

Quote:
Mac OS X 10.3 and later include read-only support for NTFS-formatted partitions...Native NTFS write support has been discovered in Mac OS X 10.6 and later, but is not activated by default, although workarounds do exist to enable the functionality. However, user reports indicate the functionality is unstable and tends to cause kernel panics, probably the reason why write support has not been enabled or advertised.
https://en.wikipedia.org/wiki/NTFS#Interoperability


My System SpecsSystem Spec
.
16 Jul 2016   #32
CurlyBen

Windows 10 Pro x64
 
 

Quote   Quote: Originally Posted by townsbg View Post
Quote   Quote: Originally Posted by CurlyBen View Post
As for file systems, I understand that different developers have differing requirements and objectives, but given that OS X can read an NTFS volume without issue it really shouldn't be a big deal to implement basic write capability. There are drivers available to allow OSX to write to an NTFS partition but they seem to get mixed reviews, because they're not properly integrated into the system. I never tried reading HFS with a Windows machine, but it shouldn't be all that complicated. It's just quite frustrating as someone who uses both systems!
Quite a bold statement to make if you don't understand OS level programming. Just food for thought. I found this on Wikipedia. Apparently enabling write support on a mac causes it to crash so perhaps Apple tried and ran into problems they couldn't fix so they avoided them altogether by disabling it (not surprising considering Apple's history). I don't think that either company is in a hurry to make their systems inter-operable considering how much money that they are already making. If you think about it you can't (without an illegal hack) put mac on a pc and most that buy a mac aren't going to put windows on it so there isn't much of a demand for them to do so.

Quote:
Mac OS X 10.3 and later include read-only support for NTFS-formatted partitions...Native NTFS write support has been discovered in Mac OS X 10.6 and later, but is not activated by default, although workarounds do exist to enable the functionality. However, user reports indicate the functionality is unstable and tends to cause kernel panics, probably the reason why write support has not been enabled or advertised.
https://en.wikipedia.org/wiki/NTFS#Interoperability
I'll admit my idea of hardware is more hammers and spanners (I'm a mechanical engineer by training... amongst other things) but it doesn't seem to be that big an assumption to say that if the OS understands the file system well enough to read it that it's not a big leap to be able to write entries for new files. There are two relatively small companies (Tuxera and Paragon) who have implemented full read-write capability on OS X, so it doesn't seem absurd to think that Apple could do it with a little effort. I understand it's not a priority for them, but it doesn't stop me wishing it was!
My System SpecsSystem Spec
16 Jul 2016   #33
CurlyBen

Windows 10 Pro x64
 
 

OK, so I've learned a little more about what the exFAT boot sector is saying (I created a 100mb exFAT partition on my new external hard drive to have a bit of a look at what's going on). It appears the first line stays the same for every exFAT partition, and the screenshots I posted yesterday are the only sectors where that string appears on the disk. Looking at the first few lines more closely, that sector isn't a working bootsector anyway as all the important attributes (hidden sectors, length, cluster size etc.) are all 0. So the bootsector is gone... but the FAT still exists to some extent, as DMDE was able to pull up the file structure when I was going through the recovery actions, although it didn't manage to recover everything successfully. I've attached a screenshot of the folder structure DMDE pulled out, and it's right up to date (the new folder structure directory was something I was working on the night before my mishap, ironically to make it easier for me to back up to a remote server via a VPN...)
Another diskpart clean tale of woe-screenshot-2016-07-15-17.42.23.png

So far my research on the exFAT bootsector makes me think it's the red 00s in lines 04 to 07 (see next image) that are the key to the bootsector, everything else is more or less generic to an exFAT partition. I'm trying to work out whether I can get enough information from DMDE to recreate how the bootsector should have looked (the final image is some data from DMDE, though I'd like to know where it's come from before using it. However that's from the volume information for the volume which shows the correct directory structure and I recovered significant amounts of data).
Another diskpart clean tale of woe-screenshot-2016-07-15-09.54.32.png
Another diskpart clean tale of woe-screenshot-2016-07-15-11.40.26.png


Anyway for the time being it's getting late, so I've rebooted my PC and I've set testdisk running another deep search (Intel/PC) with nothing else running, just in case it finds anything interesting.


My System SpecsSystem Spec
.

16 Jul 2016   #34
jumanji

Windows 7 Home Premium 32 bit
 
 

The idea of implanting the exFat bootsector from another HDD has been on the back of my mind but I had reserved it as a final option if all else fails. That idea as conceived by me:

Buy another 1TB WD Passport. It should be the same model number, same capacity so that the total number of sectors is the same.

Format it as an exfat drive creating one single volume. Backup the bootsector at 2048 using bootice.

Restore that bootsector to sector 2048 on to sector 2048 of the drive where it is missing, again using bootice.

Will that work? I have no idea. ( I formatted my drive using Windows Disk Management. It put the bootsector at 2048. Will it be the same if I had formatted it with some other utility - say using a Mac?

You are doing some good research. You had examined the template for the NTFS bootsector. Now examine the exFat bootsector template in Bootice. Try to decipher what the fields you had marked in red denote. With more active brain cells than I have , you should be able to do a better job.

If TestDisk deep search also does not find a valid partition, try PhotoRec.

Keep Zero Assumption Recovery also in mind. The digital Image Recovery module in it can be used for free. It is also geared to image recovery specific to a list of cameras. Data Recovery Software, Solutions, Tutorials - ZAR Data Recovery
My System SpecsSystem Spec
16 Jul 2016   #35
townsbg

Windows 7 pro 64-bit
 
 

Quote   Quote: Originally Posted by CurlyBen View Post
...it doesn't seem absurd to think that Apple could do it with a little effort. I understand it's not a priority for them, but it doesn't stop me wishing it was!
I agree however I think that Apple is bad at fixing problems or coming up with other solutions. After installing Yosemite I started encountering graphics issues and I've since found other problems with the OS that they ignored. They seem to be fixed in the newest version (which I put on a VM) but they didn't even attempt to fix them in Yosemite so I've had to live with them. I get the feeling that their engineers' spend most of their time developing new versions rather than fixing current ones. Pertaining exfat write support as I've stated I doubt that they consider the demand to be high enough to warrant fixing it.
My System SpecsSystem Spec
16 Jul 2016   #36
CurlyBen

Windows 10 Pro x64
 
 

Quote   Quote: Originally Posted by jumanji View Post
The idea of implanting the exFat bootsector from another HDD has been on the back of my mind but I had reserved it as a final option if all else fails. That idea as conceived by me:

Buy another 1TB WD Passport. It should be the same model number, same capacity so that the total number of sectors is the same.

Format it as an exfat drive creating one single volume. Backup the bootsector at 2048 using bootice.

Restore that bootsector to sector 2048 on to sector 2048 of the drive where it is missing, again using bootice.

Will that work? I have no idea. ( I formatted my drive using Windows Disk Management. It put the bootsector at 2048. Will it be the same if I had formatted it with some other utility - say using a Mac?

You are doing some good research. You had examined the template for the NTFS bootsector. Now examine the exFat bootsector template in Bootice. Try to decipher what the fields you had marked in red denote. With more active brain cells than I have , you should be able to do a better job.

If TestDisk deep search also does not find a valid partition, try PhotoRec.

Keep Zero Assumption Recovery also in mind. The digital Image Recovery module in it can be used for free. It is also geared to image recovery specific to a list of cameras. Data Recovery Software, Solutions, Tutorials - ZAR Data Recovery
Testdisk deep search still didn't find anything, so I've set it running using the 'none' partition table option. So far it's supposedly found a couple of HFS partitions (which I don't recall ever creating) but it will probably be running for another 24 hours or so yet. I had a quick go with Photorec but it was behaving a little strangely - at first attempt (with default file options) it was finding the CR2 image files but also a huge volume of irrelevant .apple and .plist files, so I cancelled it and just selected the file types I'm interested in. I tried a few different combinations but it was finding almost nothing, so I've abandoned that for the time being - I may try again when I have another drive large enough to extract all the files to.

I also had a look at Zero Assumption Recovery but it needs a partition to search, so I'll leave that for the time being.

I like the idea of getting another 1tb Passport drive to experiment with. I'll have a look for one.

I've got a reasonable idea so far of what the fields marked in red are. I've put my comments in italics after each one:
  • Hidden sectors
    The number of sectors prior to the bootsector, i.e. for the first bootsector 2048 (sectors 0-2047)
  • Total sectors
    For a disk with a single partition presumably total sectors - hidden sectors. I suspect nothing will go wrong if this value is too large
  • FAT offset
    Not sure yet whether this is a fixed or changing number
  • FAT size
    This could be difficult - my understanding is the FAT will increase in size as more files are added, so the FAT size of a new partition will be far smaller than that of a partition with hundreds of thousands of files
  • First cluster offset
    Not sure on this yet
  • Clusters count
    Not sure on this yet
  • Root directory first cluster
    Not sure on this yet
  • Volume serial number
    I think this is just a way of uniquely identifying the volume to the OS, so probably doesn't need to match the original
  • File system revision
    Apparently the only current revision is 00 01
  • Bytes per sector
    Will almost certainly be the default value. 512 according to DMDE
  • Sectors per cluster
    As above. 256 according to DMDE
  • Number of FATs
    I believe exFAT only uses a single FAT
  • Media descriptor
    Describes the type of drive. Can almost certainly be left at 80 (the value used in my trial exFAT partition)

So that's where I've got to so far. My thoughts on where to go next:
  • I can probably get hold of a couple more exFAT bootsectors from drives my brother has. I need to get a better understanding of which values change between partitions and which don't
  • I'm going to get another 1tb Passport drive and format it as exFAT as suggested. I'll back up the bootsector then copy all the recovered files to it and look to see how the bootsector changes, in particular the FAT size. I'll also experiment with formatting from OS X and see whether it creates a different bootsector to Windows 10.
  • I'll try and find out how DMDE has come up with some of it's information, as it's clearly found the FAT, despite not being able to correctly recover everything
  • When I've got enough information together to have a go at creating a new bootsector I'll copy it to my existing drive, then run Testdisk. I assume that if I've got the new bootsector correct (or at least close) I'll be able to list the contents of the disk before anything other than sector 2048 has been modified. If it doesn't work simply zeroing sector 2048 should return the drive to its current state

Any suggestions or criticisms welcome. I'm probably not going to be able to look at this much for the next week or two as I've got my flight instructor exams coming up and I should be doing some more studying! Although whilst I'm really annoyed at myself for causing all this hassle, I am quite enjoying learning a bit more about how the system works and the challenge of recreating it. That will turn to pure annoyance when the progress stops!
My System SpecsSystem Spec
16 Jul 2016   #37
CurlyBen

Windows 10 Pro x64
 
 

Quote   Quote: Originally Posted by townsbg View Post
Quote   Quote: Originally Posted by CurlyBen View Post
...it doesn't seem absurd to think that Apple could do it with a little effort. I understand it's not a priority for them, but it doesn't stop me wishing it was!
I agree however I think that Apple is bad at fixing problems or coming up with other solutions. After installing Yosemite I started encountering graphics issues and I've since found other problems with the OS that they ignored. They seem to be fixed in the newest version (which I put on a VM) but they didn't even attempt to fix them in Yosemite so I've had to live with them. I get the feeling that their engineers' spend most of their time developing new versions rather than fixing current ones. Pertaining exfat write support as I've stated I doubt that they consider the demand to be high enough to warrant fixing it.
I'll happily admit my experience of using an Apple computer has been well below what I was expecting. There appears to be an issue in El Capitan where running a virtual machine from a USB drive results in exceptionally slow running. This is a bit of an issue for me as I need to run Windows for some CAD software and a few other programs, and I only have a 120gb SSD due to the hideous prices Apple charge for storage upgrades. Oh well! There's a lot I like, but I'm certainly not an Apple evangelist.
My System SpecsSystem Spec
16 Jul 2016   #38
CurlyBen

Windows 10 Pro x64
 
 

Thinking about how I can work out the unknowns as listed above, it seems the thing to do is work out what the FAT should look like and then search for that. Then it doesn't really matter whether bootsector is identical to the one that's been lost, provided the FAT offset etc. are correct relative to the new bootsector. I think the best way to do that is going to be to create a new exFAT partition, completely zero it, then add some files to it and identify the FAT and its components. That's going to require a little more time than I have right now, but I'll post back when I've done so in case anyone is interested/attempting to go through the same process!

PS I'm confident the FAT is still there and at least mostly intact due to DMDE finding the correct folder structure.
My System SpecsSystem Spec
20 Jul 2016   #39
jumanji

Windows 7 Home Premium 32 bit
 
 

Returning to this thread after many days, since I stumbled on something that may interest you.

Another diskpart clean tale of woe-20-07-2016-20-40-05.jpg

https://digital-forensics.sans.org/s...-exfat-ham.pdf -exFAT System revealed and dissected.

According to this, if the first Volume Bootsector is at 2048, the backup of sectors 2048 to 2059 will be there
at sectors 2060 to 2071

I have checked this on my trial volume examining sectors 2048-2059 and 2060-2071 with bootice. ( The trial HDD has just been quickformatted as exFAT). Sector 2048 which is the first sector of the volume is backed up in sector 2060. In my trial drive sectors 2049-2058 were all zeros as also sectors 2061-2070.Sector 2059 had some repetitive data strings and sector 2071 had the same repetitive string.


My System SpecsSystem Spec
20 Jul 2016   #40
CurlyBen

Windows 10 Pro x64
 
 

Thanks jumanji, that link is interesting. I've just acquired another drive (not a 1tb unfortunately, but it will help) and I'm just in the process of cleaning the whole disk to make sure there's nothing there to confuse things. At the moment I'm struggling to work out what the FAT offset value should be - on my first trial partition the bootsector is at 2048 and the FAT offset is also 2048, which suggests to me the FAT should start at 4096, but the drive is blank between 4000 and at least 8000. I'll check the old drive for backup bootsectors but I'm fairly sure they're gone, as I searched for the first line which should be common to all exFAT bootsectors and I only found it once and that was a blank bootsector anyway. The FAT seems to start somewhere around sector 60,000 (I haven't worked out exactly where it starts) so it's possible that partition was slightly offset from the start of the drive anyway (clearly it wasn't already complicated enough!)
My System SpecsSystem Spec
Reply

 Another diskpart clean tale of woe




Thread Tools Search this Thread
Search this Thread:

Advanced Search




Similar help and support threads
Thread Forum
Disk - Clean and Clean All with Diskpart Command
How to "Clean" or "Clean All" a Disk with the Diskpart Command This will show you how to use the clean or clean all command on a selected disk to delete all of it's MBR or GPT partitions, volumes, and any hidden sector information on MBR disks is overwritten. The data on the HDD is not...
Tutorials
Diskpart clean done accidently...Please help!!
Hello There, As I read through other diskpart clean discussions... I also have done the same and I run a scan with partition recovery wizard and it is showing my data..but I don't know how to recover it. I contacted with minitool partition recovery wizard help, They said to do a scan with...
Backup and Restore
Formating 3tb drive..diskpart clean or clean all
I am having to format a 3tb drive that I am installing Windows 7 on and I need to know do I need to do clean all or just clean in diskpart? What happened is I did not know about the GPT (first time dealing with a drive over 2tb) and I just went through the motions like you do with installing. Well...
Installation & Setup
Should I clean my HDD with Parted Magic or Diskpart clean all command?
Hi everyone i have a question about HDD Clean and also question about Win 7 64 bit :) I want to wiped out my all HDD data inside i want to delete all information like a brand new HDD. I heard that there is a program called Parted Magic can do this and also Diskpart clean all command too. ...
General Discussion
Do not see HDD after clean all using diskpart
Can someone please help with this. after do clean all follow diskpart and format HDD. now do not see this hdd in my computer but see it in DISKPART. thanks,
General Discussion
Diskpart Clean All Error
Hello, came here to post my problem if anyone is willing to help. Due to a recent virus which I can't seem to get rid of, I'm going to do a clean install of windows 7 from an iso file with virtual clone drive. However, after researching a bit I found it would be best to first do a clean all...
General Discussion


Our Sites

Site Links

About Us

Find Us

Windows 7 Forums is an independent web site and has not been authorized, sponsored, or otherwise approved by Microsoft Corporation. "Windows 7" and related materials are trademarks of Microsoft Corp.

Designer Media Ltd

All times are GMT -5. The time now is 09:32.

Twitter Facebook Google+



Windows 7 Forums

Seven Forums Android App Seven Forums IOS App