Quickest Way To Move 160GB?

bobtran said:

Not quite accurate...compressing data on the box prior to transmitting over data lines will be faster than sending uncompressed data over a usb or even broadband depending on size of file. I agree that for small files there will not be any savings (and it may take longer to compress/decompress) but as files get larger the savings multiply. compression can result in 30%-50% smaller files, (sometimes even more) unless the files have fixed field length date fields instead of variable length data fields, which can substantially reduce the time/bandwidth required to move a file. Running a compression or decompression natively is extremely fast unless you are using a really old box. Furthermore you don't need to decompress the files at storage end unless you want/need access to the contents there.

I do agree that the easy way is to send it native (The original post specified fastest way to send 160gb, not ease of process).

By way of manly experiment...

Source: [local drive] W:\MX\*.* (about 4.6GB worth of various media files)
Dest: [local drive] V:\
Machine: unadulterated Q6600 with 8GB of RAM running W7x64

First, a straight robocopy, uncompressed:

Code:

 
W:\>robocopy w:\mx v:\mx *.* /E /MT:8
...
Total Copied Skipped Mismatch FAILED Extras
Dirs : 35 35 0 0 0 0
Files : 831 831 0 0 0 0
Bytes : 4.683 g 4.683 g 0 0 0 0
Times : 0:21:53 0:02:09 0:00:00 0:00:16

Two minutes and nine seconds for the copy.
Effective throughput without compression: ~37 MBytes/sec.

================================================

Now using (command-line) 7-zip to first compress those same data with the default compression settings:

Code:

 
C:\Program Files\7-Zip>7z a -r w:\ w:\MX\

It took 27 minutes (frickin' thing doesn't do stats by default and I couldn't be bothered doing it again) to produce this archive:

Code:

 
W:\>dir .7z*
 Volume in drive W is Mirror
 Volume Serial Number is A4F5-EDC0
 Directory of W:\
16-Oct-09  12:07 PM     4,911,076,674 .7z
               1 File(s)  4,911,076,674 bytes
               0 Dir(s)  32,615,526,400 bytes free

Now copying that across to the second drive, again using Robocopy:

Code:

               Total    Copied   Skipped  Mismatch    FAILED    Extras
    Dirs :         1         0         1         0         0         0
   Files :         1         1         0         0         0         0
   Bytes :   4.573 g   4.573 g         0         0         0         0
   Times :   0:03:21   0:00:50                       0:00:00   0:00:50

Total time taken: 27min + 0:50 = 27:50
Effective throughput with compression: ~2.8 MBytes/sec

The compression strategy would be worthwhile if we were traversing a "wet string" network link. For local-to-local copies it's counter-productive. My technique may be questionable, but even if the "with compression" results were an order of magnitude better they'd still fail to beat the uncompressed copy.

Quickest way to move it you say....

Just lift up the HD from original spot, and place it down in new spot!

I know, sorry couldn't resist.

bobtran said:

27 min to compress the 4.6gb file? I think something is wrong there.

Yeah, it was downright painful. I don't think it's broken though. 7zip just seems rather aggressive in its default compression, and the fact that the content was not particularly compressible probably didn't help.

bobtran said:

... although I think gzip is a better compression tool.
I DO use 7zip myself for normal zip/unzip and find it a good and worthy tool.

I picked 7zip for the test mostly for its independence from Windows itself, and for its good reputation. I doubt any compression utility could do it 20 times faster, which is what it would take to approach the uncompressed throughput.

bobtran said:

Thanks for making me check this out, having a hard time talking around my foot....I learned something tonight. I will have to check out the robocopy product although I get those speeds without it.

Thank you for an interesting discussion :)

Robocopy is built into the OS. That by itself is worth a lot to me, given my own typical usage patterns.

Hi all

Have I missed something here - but why not use something like Acronis - it will back up FILES as well as entire Disk Images (partitons).

1) start TI.
2) Select Data
3) click against folders you want to copy
4) backup to external HDD using with highest compression ratio
5) restore to target disk

job done -- won't take that long either.

Cheers
jimbo