Solved Question about format cluster size

[Golden]

G'day,

I've been experimenting with EASEUS Partition Master Home Edition and a spare 500GB WD 5000AAKS external USB disk as part of my self education. Specifically, I've been experimenting with cluster sizes after reading this in the EASEUS help file:

EaseUS Partition Master Manual - Formatting Partitions with partition manager software, unformat hard drive with format recovery software.

The smaller cluster size is, the bigger file allocation table (FAT) will be. The bigger the FAT is, the slower the operation system works with the disk.

Perhaps I misunderstood, but does this imply that using a larger cluster size will cause the disk to be accessed faster?

To test this, I formatted the WD as NTFS using a 512 byte cluster size and then timed a copy of 3.1GB from my internal HDD to the WD USB drive. I repeated the exercise after formatting a second time using a 64KB cluster size.

These are the results:

512 byte cluster size : time for copy = 5min 50sec
64KB cluster size : time for copy = 6min 41sec

The result seems counter-intuitive given my understanding of the line from the help file. Have I misunderstood? What is the benefit of formatting using different cluster sizes? Why is there options to format using different cluster sizes?

Thanks,
Golden

 

[Corazon]

Good question. Did you copy one single 3.1GB file or many smaller files totaling 3.1GB? That would be an important difference.

Smaller files on a disk with a large cluster size will obviously use more space and I'm guessing that it would take longer to write them because there'll be much more padding at the end of such files in order to fill them up to multiples of the cluster size.

Example: your cluster size is 512 bytes. You write a roughly 17K file. No problem, as the file will occupy 34 or maybe 35 clusters and fit nearly perfectly.

With a cluster size of 64KB, you end up with the 17KB file occupying one cluster and the remaining 47KB having to be padded with zeroes by NTFS - slowing down the file transfer in the process.

 

[Golden]

Thanks for the reply Corazon.

The 3.1GB was comprised of 411 files over 8 sub-directories ranging in size from 1KB, through to the largest file which is 1.2GB. Most of the files are in the 30KB to 500KB size, with about 40 odd files ranging in size from 1MB through to 200MB.

Regards,
Golden

 

[ignatzatsonic]

I did something similar the other day.

I have an 8 gig USB drive to which I periodically copy about 30,000 files totaling about 5 gigs. That's about 166k per file on average. About 2/3 of them are JPGs averaging about 80k.

The USB stick is FAT32 and had the default cluster size---4096k I think. Formatted that way, the 5 gigs copied in a little over 3 hours.

I reformatted the USB stick using diskpart and gave it a cluster size of 32k, still FAT32. The 5 gigs copied took 3 hours and 20 minutes. Very little difference compared to 4096k clusters.

I'm not sure what it proves.

A year or so ago, out of curiosity, I ran a tool that measured slack space on my hard drives, all of which are NTFS with default cluster size. The pleasantly surprising results:

My C drive had 199 MB of slack out of a total of 60 GB space
My D drive had 101 MB of slack out of a total of 536 GB space
My E drive had 285 MB of slack out of a total of 1.36 TB space

D and E had less than 1/10 of 1% slack.

The C drive had about .3% slack on the 60 GB partition.

Total slack was only 585 MB.

 

[Corazon]

The USB stick is FAT32 and had the default cluster size---4096k I think. Formatted that way, the 5 gigs copied in a little over 3 hours.

I reformatted the USB stick using diskpart and gave it a cluster size of 32k, still FAT32. The 5 gigs copied took 3 hours and 20 minutes. Very little difference compared to 4096k clusters.

Interesting results, but likely misleading too. The big question is if the partition on your USB stick is correctly aligned (for example, starting at sector 64)? Sadly, all too often they start at sector 63 (stupid legacy thing) so the clusters are completely misaligned with the internal block structure of the stick (same reason why partitions on SSDs and "advanced-format" HDs with 4096 bytes per sector must be aligned, or performance is severely impacted).

 

[ignatzatsonic]

Interesting results, but likely misleading too. The big question is if the partition on your USB stick is correctly aligned (for example, starting at sector 64)?

Yeah; I checked that in diskpart. It's aligned beginning at 64.

Probably a meaningless experiment. I was just curious.

It's a 3 year old Kingston drive I use for ad hoc data backups.

 

[alan10]

Probably a meaningless experiment. I was just curious.

I suspect superbly meaningless.

Writing files to disc is a meaningless exercise if you never read them

It would be far better to optimise the READ access of typical files.
e.g. a few minutes after reboot use a simple CMD.EXE action to DIR list contents of C:\Windows\system32 sorted on ACCESS date.
Copy those files a test folder.

Format your spare HDD and copy the test folder to it.
Copy all files from the spare HDD to NUL and measure duration.
Select the test folder copy, look at properties, and observe "Size on Disc" which is worse with smaller files and larger clusters
Repeat with different format and again measure duration.

That will indicate which format has the worst effect on start-up time and how much disc space is used.

 

[ignatzatsonic]

Probably a meaningless experiment. I was just curious.

I suspect superbly meaningless.

Writing files to disc is a meaningless exercise if you never read them

It would be far better to optimise the READ access of typical files.

You are quoting me, so I guess you are talking to me.

I hope to never have to read the USB drive in question. The only way I would need to read it is if I had a hard drive failure and had to use that USB stick to restore. I don't much care about its read speed.

On the other hand, I do write to that drive regularly--every few weeks. So I was wondering if a change in cluster size would improve the writing speed for my set of files to that specific drive. As it turns out, it has little effect.

Your mileage may differ.

 

[alan10]

You are quoting me, so I guess you are talking to me.

Sorry. What I actually had in mind was the O.P. test of measuring how long it took to write, instead of what to me is more relevant, the impact of Read duration and Disc Space used.

 

[Zenicablues]

This is question. not reply: What this mean. before boot menu, every time I get, WARNING! FAT CLUSTER SIZE (=65536) LARGER THAN 32K! Stay about 0.5 sec an go to boot menu ! It's not see terrible, but is very annoying !!! and what FAT SIZE in NTFS disk !!! Can anybody to explain me, thankfully in advance !!! Alex (ZenicaBlues)