Windows 7: Corruption in large file transfers to USB drives

I think what we are saying is that maintaining file integrity isn't much of an issue that any of us have seen with Win7, so the actual problem might be the way the file checking program interacts with Win7 causing it's output to be possibly faulty.

Sometimes I copy multiple ISO's, extract, recompile dozens of times and have never had any file degradation with Win7. The only need to check file HASH was when the download source was questionable, but never after it got onto Win7.

What I would do next in your situation is go to the website for the file checking program and find out the best way to approach them - forum, email, chat, phone - then ask for a definitive answer to your problem.

First of all, I think we agree here that we are dealing with a rather rare problem and thus we need to pinpoint exactly what's going on and what's the reason for it.

Here's what I would do. As far as I am concerned, there are two types of files - text files and binary files. Both types of files can be directly compared. Why don't you take a case where your testing software reports corruption and directly compare the copied ("corrupted") file to the original and see what the difference is. You can use Windiff, Merge, or other similar tools to do that. With text files it's probably easier, but I am not sure what kind of files you're talking about.

Could you make a test comparison, document what you did - how the files got corrupted in the first place - and post the actual difference between them? I will then try to reproduce the problem on my own machine.

Bill2 said:

Try copying with a 3rd party tool like teracopy.

Thank you Bill2 for recommending Teracopy, I actually had the time to test this program yesterday and it works like a charm. Spent 3 hours copying files to external hard drives and not one single incident of corruption. It even has the option to do the CRC checking for me after completing the copy. I used my usual programs to check the CRC as well and they all passed.

I have a brandnew laptop (Asus G53JW), where the same usb problem occurs, data loss/corrupted data while copying/moving large files with win explorer from Win7x64 to an external NTFS formatted USB harddrive.

I found the behaviour accidentally while creating images of the preinstalled recovery partition with sizes bigger 10 GB. Splitting the files in smaller chunks of 1G didn't help either.

The problem was present from the beginning, with the preinstalled OS (Windows7x64 Home Premium). The two internal SATA drives have no problem when copying with explorer from one disk to the other.

I installed the laptop completely from scratch using an genuine MS installation dvd, later all updates until SP1 and every single most recent driver from the manufacturer without any problem, even the BIOS is updated. Everything works fine, just the old problem there - again. Until this time I avoided installing FW/Virus software that could interfere the usb transfers.

By trial and error I found, that XCOPY produces no errors on copies in a CMD Window by some reason, but only when invoked with the /J switch, it disables i/o buffering: "xcopy /J <sourcefile> <destination>".

BTW Linux/Knoppix copies with perfection even under its GUI.

Switching the drive's write cache on/off doesn't change the situation under Win7.
The hardware used by the laptop is a i7 740QM, HM55 Express Chipset and a NVIDIA GTX460M.

Another interesting thing is that the System offers 2 seperate USB Host controllers, one by intel and one by fresco with different drivers, but the problem appeared on all USB hardware.

I guess the reason is more an internal OS mechanism with the win explorer, maybe with buffers or process syncronisation rather than a hardware or driver failure.

But honestly, I have absolutely no clue which screw to turn in Win7, for example how can I switch off i/o buffering in general for external usb drives?

I hope that some of the experts here have an idea, thanks in advance.

Cheers
Riley

I'm also experience problems transferring large files to external usb hard drives.

My computer uses Windows 7 Home Premium 64 bits, my usb controllers are from Intel and the hard disk is a external 1Tb NTFS formatted.

The corruption for me started when the drive was about 50% of its capacity. The corruption consists in random blocks of zeros in the file.

These blocks of zeros are 20480 bytes in length. None of the files places at the first half of the disk were corrupted, but many on the second half was. Some files as small as 200 Mb were corrupted.

After intense hard drive checking and surface scan I discarded physical problems on the hard disk.

Using xcopy /j does not work, neither disable antivirus or 3rd party utilities.

I have reformated the hard drive to FAT32, and up to date, not a single byte corrupted. The main disadvantage is the need to split very large files, but I prefer this small oddity that loss of data.

For me it is clear there is a serious issue with external usb NTFS drives with Windows 7.

Hope this helps someone to find the problem.

Hi everyone, I'm having the same problem and it's very annoying...
I experienced file transfer corruption with large files from two different laptops (both HP, a dv6000 and a dv5, with Windows 7 Ultimate x64) to two different external hard drives (a WD passport 1TB and an Iomega 500GB), both formatted NTFS.
When using Teracopy I'm not having crc errors.
Did anyone found a solution?

Yes, I updated to SP1 (and all the latest updates) both PC.

Hate to revive a thread as old as this one, but was there ever a solution found for this?

I work in video and have noticed frames missing/blank/corrupted in my editor sometimes. I play back the file directly in VLC, and I see there is a corrupted frame in that location. The thing is, the original file on the storage card media does not have this missing frame data at all. I downloaded an MD5 checksum application and compared the files to the original card. Sure enough, some files had matching checksums, and some did not (the ones that did not happen to be the ones with the corrupted data).

I thought maybe my card reader was going bad, but I just archived a large chunk of data from another USB drive, and lo and behold: same type of error popped up. I started thinking, maybe my USB drive is going bad? But CHKDSK and all other tests performed on it come back clean. I tried googling this problem, and imagine my surprise to have found this and many other threads discussing some sort of issue with Win 7 64 bit!

I'm beyond shocked that this has been an issue all along and I never knew about it. I blamed every other piece of hardware to no avail, when the problem was actually Windows itself!

Has anyone found a solution? I see in another thread someone mentioned his RAM had gone bad, but it is confirmed in this one that this isn't the issue for the OP.

...yes, I'm still running Win7 64 bit. I've got Win10 on some of my other personal computers, but my work machine has some legacy tools for media editing that isn't supported anymore. It took me a long time to set up properly and I don't want to fix what isn't broke by going to a new OS only to discover new problems. But not being able to trust Windows copy commands? This is a HUUUUGE ISSUE!