Windows 7 Forums
Welcome to Windows 7 Forums. Our forum is dedicated to helping you find support and solutions for any problems regarding your Windows 7 PC be it Dell, HP, Acer, Asus or a custom build. We also provide an extensive Windows 7 tutorial section that covers a wide range of tips and tricks.


Windows 7: File Compression

22 Jan 2011   #11
cluberti

Windows 10 Pro x64
 
 

Quote   Quote: Originally Posted by seekermeister View Post
When I just formatted the 500GB drive, I selected the option to use file and folder compression. I wasn't certain if that was the thing to do, but I tossed a coin. Now, I'm wondering just how much it is able to compress a file...anyone know?
Yes, but I'm not sure you understand just how technical a question you just asked . NTFS compression uses something called a "compression unit" to determine the alignment and size of compressed byte ranges in a data stream (aka, a file or file segment). The default NTFS cluster size (if Windows formatted your drive) is 4K, so the CU size on a disk using those cluster sizes is 64K (16 clusters x 4K) - note that a volume with a cluster size larger than 4K cannot use NTFS compression (it can still do sparse files, but not NTFS compression).

Anyway, NTFS takes the file and compresses it - this is the actual compression - and then divides the file up into the compression unit. If the compression of the file results in the compression unit (again, default of 64K) being one or more clusters smaller than it was originally, that compression unit is written to disk compressed. If that section of the file stream does not compress smaller than 1 or more cluster, it's written to disk as-is, uncompressed. This makes for retrieval faster (parts can be uncompressed, and only the compressed parts need to be decompressed if needed) in random segments, but of course does make sequential transfer slower because you are going to have to decompress the compressed file stream units to copy the file.

Remember that NTFS compression is actually done at the file system allocation unit, but the type of data in a file still does matter - uncompressable data is still uncompressable data. If you have a movie file that doesn't compress very much when run through the high compression modes of WinRAR or 7zip, for example, the filesystem compression isn't going to get you much further than that either. NTFS compression is actually tuned for speed, not efficiency - keep that in mind when deciding if compressing your data at the filesystem would be any better than using a compression utility.

Another hint - movie files make *bad* compressed files - meaning, you're not likely to get much compression, if any, from a standard movie file. Either it's lossless and has little "empty" data regions, or it is a lossed container and is quite compressed in the file already, and as such is also not very "empty" either. Also given that a movie generally doesn't have the same byte pattern over and over, it doesn't compress well. You'd be better buying a larger drive as temporary storage, or see if the vendor will "cross-ship" you the new drive - you can wait until you receive the new drive (and copy data to it) before you have to send the old one back.


My System SpecsSystem Spec
.
22 Jan 2011   #12
kado897

Microsoft Community Contributor Award Recipient

Microsoft Windows 7 Home Premium 64-bit Service Pack 1
 
 

Quote   Quote: Originally Posted by seekermeister View Post
Quote   Quote: Originally Posted by kado897 View Post
The problem is whether they would compress enough to fit on the temporary drive without loss of quality. The normal archive formats such as zip and lzh are all lossless compression methods which would extract to the original quality. These types of archive hardly compress videos at all. The only way you can really make a video smaller is to reduce its quality. Once you have done that you can't get it back again.
I understand what you are saying, when it comes to the manner in which a video is originally wrapped. But I'm not talking about changing their original state, and not about using a zipper, but a compression method that is external to the files, such as I mentioned in formatting the drive, which is used by the OS, or a backup method used by something like True Image, which would also be external to the files. It is possible that neither of these methods would compress the files at all, if that is the case, then that is the answer to my question. If they would compress the files...as a group, rather than individually, would not the original quality be retained, after removing them from their compressed enviresonment?
I have to admit that I don't know how that compression works. I suspect the amount of compression would depend on how much disk space the NTFS file system wastes. I don't think it would be very much but you are correct in that you would not lose any quality.
My System SpecsSystem Spec
22 Jan 2011   #13
cluberti

Windows 10 Pro x64
 
 

Correct, but the compression ratio would likely be less than 1%. See my previous post.
My System SpecsSystem Spec
.

22 Jan 2011   #14
kado897

Microsoft Community Contributor Award Recipient

Microsoft Windows 7 Home Premium 64-bit Service Pack 1
 
 

Quote   Quote: Originally Posted by cluberti View Post
Correct, but the compression ratio would likely be less than 1%. See my previous post.
Our posts overlapped. That's about what I would have guessed.
My System SpecsSystem Spec
22 Jan 2011   #15
seekermeister

W7x64 Pro, SuSe 12.1/** W7 x64 Pro, XP MCE
 
 

Quote   Quote: Originally Posted by cluberti View Post
Yes, but I'm not sure you understand just how technical a question you just asked . NTFS compression uses something called a "compression unit" to determine the alignment and size of compressed byte ranges in a data stream (aka, a file or file segment). The default NTFS cluster size (if Windows formatted your drive) is 4K, so the CU size on a disk using those cluster sizes is 64K (16 clusters x 4K) - note that a volume with a cluster size larger than 4K cannot use NTFS compression (it can still do sparse files, but not NTFS compression).

Anyway, NTFS takes the file and compresses it - this is the actual compression - and then divides the file up into the compression unit. If the compression of the file results in the compression unit (again, default of 64K) being one or more clusters smaller than it was originally, that compression unit is written to disk compressed. If that section of the file stream does not compress smaller than 1 or more cluster, it's written to disk as-is, uncompressed. This makes for retrieval faster (parts can be uncompressed, and only the compressed parts need to be decompressed if needed) in random segments, but of course does make sequential transfer slower because you are going to have to decompress the compressed file stream units to copy the file.

Remember that NTFS compression is actually done at the file system allocation unit, but the type of data in a file still does matter - uncompressable data is still uncompressable data. If you have a movie file that doesn't compress very much when run through the high compression modes of WinRAR or 7zip, for example, the filesystem compression isn't going to get you much further than that either. NTFS compression is actually tuned for speed, not efficiency - keep that in mind when deciding if compressing your data at the filesystem would be any better than using a compression utility.

Another hint - movie files make *bad* compressed files - meaning, you're not likely to get much compression, if any, from a standard movie file. Either it's lossless and has little "empty" data regions, or it is a lossed container and is quite compressed in the file already, and as such is also not very "empty" either. Also given that a movie generally doesn't have the same byte pattern over and over, it doesn't compress well. You'd be better buying a larger drive as temporary storage, or see if the vendor will "cross-ship" you the new drive - you can wait until you receive the new drive (and copy data to it) before you have to send the old one back.
The issue is not how technical of a question that I asked, but how technical of an answer that it receives. Your's seems to do well on this account. If the compression ratio is only 1%, as you said in your followup post, then it is obvious that I'm beating a dead horse, and need to find a new mount.

Since I doubt that WDC will cross-ship drives, I would need a new one, and the choices are extremely limited meeting my criteria. I have found only two, another like I have, which I'm not very enthusiastic about, or a Seagate twice the size, and twice the price. I would still consider that, if I had confidence that their drive was reliable, but with my recent experience with their 7200.11 drives, I don't have that confidence.

My only other options are to either format the drive that W7 resides in on my secondary computer, for a backup, or divide the videos into smaller directories for storing in multiple locations...I don't like any of my options.
My System SpecsSystem Spec
22 Jan 2011   #16
kado897

Microsoft Community Contributor Award Recipient

Microsoft Windows 7 Home Premium 64-bit Service Pack 1
 
 

I take it that these videos are not backed up then.
My System SpecsSystem Spec
22 Jan 2011   #17
seekermeister

W7x64 Pro, SuSe 12.1/** W7 x64 Pro, XP MCE
 
 

Not yet, that is what I'm trying to accomplish. Since I don't have a readily available HD large enough to back them up, the only other ways that I know of is to back them up to DVDs, but that would require ~150 disks for the job, or to use an online storage, but that would take an eternity to upload/download, and I don't know that any would be large enough for 750GBs of data.

EDIT: At the moment, I wish my Blu Ray player could write, but it doesn't. I guess a person could get some 50GB BD disks and write the videos on them. However, with the price of BD writers and media, I'm not going to do that.
My System SpecsSystem Spec
22 Jan 2011   #18
kado897

Microsoft Community Contributor Award Recipient

Microsoft Windows 7 Home Premium 64-bit Service Pack 1
 
 

Quote   Quote: Originally Posted by seekermeister View Post
Not yet, that is what I'm trying to accomplish. Since I don't have a readily available HD large enough to back them up, the only other ways that I know of is to back them up to DVDs, but that would require ~150 disks for the job, or to use an online storage, but that would take an eternity to upload, and I don't know that any would be large enough for 750GBs of data.

EDIT: At the moment, I wish my Blu Ray player could write, but it doesn't. I guess a person could get some 50GB BD disks and write the videos on them. However, with the price of BD writers and media, I'm not going to do that.
Tricky but then I am paranoid about backups. I suppose that disks are more reliable than the used to be but...
My System SpecsSystem Spec
22 Jan 2011   #19
SledgeDG

Windows 7 Ultimate x86
 
 

Every Packer/Archiver allows you to split up bigger files into several small archives (which after extraction turn back into one large untainted file) But unlike on the pics you would want to set the compression to low/store when it's about precompressed material the few % of gain wouldn't justify the time used.

7-Zip
File Compression-capture.jpg
WinRAR
File Compression-capture1.jpg

WinRar has even another helpful feature: it can add some sort of a checksum (recovery record) to the archives which helps you to repair the ones that might have gotten corrupted during file transfer

Alternatively you could try HJSplit which doesn't compress at all but splits every file into a number of smaller chunks
File Compression-capture2.jpg


Edit Nah...forget this...750 GB it's not a size my recommendations are ideal for


My System SpecsSystem Spec
22 Jan 2011   #20
seekermeister

W7x64 Pro, SuSe 12.1/** W7 x64 Pro, XP MCE
 
 

Quote   Quote: Originally Posted by kado897 View Post

Tricky but then I am paranoid about backups. I suppose that disks are more reliable than the used to be but...
I don't know that disks are any more reliable. Even though you can verify the writes, I still have had some that didn't work, and also you have to consider the possibility of a disk getting damaged. I prefer the idea of backing up to a hard drive. The hard drive could go bad, but the odds aren't too great of two drives going bad at the same time (unless you have my luck).

EDIT: I just thought of another possibility...I have another Seagate 500GB 7200.11 in the closet, that needs to be RMAed, but since I received a series of bad replacements from Seagate, I had resigned it to the closet instead. I suppose that even if I got another bad replacement, it might last long enough so that I could split the video archive in half between two drives, and the only cost would be the shipping...I don't know.
My System SpecsSystem Spec
Reply

 File Compression




Thread Tools




Similar help and support threads
Thread Forum
Check Compression Level In An Existing .rar File?
Hi all, Just wondering if there's a way to check the compression of an already rared .rar file. I'm trying to create a profile within WinRar and want to ensure it's doing exactly what I want it to do as far as compression (Store as opposed to the other compression levels). I don't want...
Software
What's your CPU compression rate
We have the mem assesement thread so I thought I would start this one. Run command prompt as admin, enter winsat cpu -compression and post your results. Here is mine.
Performance & Maintenance
Do you have a prefered file compression software or file format?
Iuse winrar and need to know if there are any differences between the different compression formats ( zip, rar, tar) and if one of them is more robust than the rest?
Backup and Restore
compression ?
hey once i've compressed a file, how do i uncompress it? thanks
Backup and Restore
Image Compression
I haven't been backing up my video collection, due to it's size (~1565GBs), because I didn't want to dedicate that much hard drive space to that task, but after looking at the compression options in TI, that could be reduced to ~898GBs, so that it would fit on a 1TB drive, with some room to spare. ...
Backup and Restore
Files Compression
is it okay to compress files/folders? is it advisable to do that on Program Files, maybe?
General Discussion


Our Sites

Site Links

About Us

Find Us

Windows 7 Forums is an independent web site and has not been authorized, sponsored, or otherwise approved by Microsoft Corporation. "Windows 7" and related materials are trademarks of Microsoft Corp.

Designer Media Ltd

All times are GMT -5. The time now is 10:19.
Twitter Facebook Google+ Seven Forums iOS App Seven Forums Android App