Syncing folders that have renamed files and subfolders

catalogue · 17 Oct 2015

This question is about syncing and backing up files in general. Sync Toy and a few other backups tools can not do what I need.

The problem is how to synchronize folders of thousands of photos and music files between my computer and external HD. I am constantly renaming files, adding and deleting child and parent folders, moving photos or folders to new folders etc. on the computer. So I don't see how any sync program could correctly deal with files and subfolders that have been moved to different folders. When any given folder on the original and backup is compared, the program can't tell the difference between a file that is missing on main computer because I wanted to delete it permanently and one that was missing because I moved it to another folder or renamed it - correct?

Yes there are programs that show you comparisons between two folder but that does nothing to solve the problem - you still have to go through everything manually and handle the missing-vs-moved-vs-renamed situations yourself, one by one.

So now imagine this happening for hundreds and hundreds of files and subfolders every week.

It seems the only solution would be delete all 200 Gb of music and photos from the HD and re-copy them every time. Or, use continuous live cloud backup but that is absolutely not an option for me. First of all I tried it and it slowed down the computer outrageously - even though I was only backing up a few hundred Gb of text files and NOT my 200 Gb of photos and music! Secondly it's not acceptable due to security and privacy issues. Either issue alone is enough to rule out cloud storage.

ignatzatsonic · 17 Oct 2015

I don't see why you should have a problem.

I constantly rename, add, delete, and otherwise reorganize tens of thousands of mp3s, pictures, and video files across hundreds of folders on my data drive.

My backup program is configured to do exactly one thing: make the backup (target) drive be an exact replica of the original (source) drive. It does exactly that.

If I move dog.jpg from the pet folder on the source drive to the animals folder on the source drive, that change is carried over to the target drive the next time I run the backup. That is: dog.jpg is deleted from the pet folder on the target drive and added to the animals folder on the target drive.

Ditto for any renamed, deleted, or newly created files on the source drive.

Backup apps have a configuration setting that controls this. It's usually referred to as "mirroring". If you do not choose mirroring, the dog.jpg file would be copied to the animals folder on the target, but it would NOT be deleted from the pet folder on the target. You would then have 2 dog.jpg files on the target. Some people want it that way and some don't. That's why there is a configuration switch for mirroring. You choose yes or no to the mirroring option. I choose yes.

What am I not understanding about your situation?

I use FreeFileSync, but as far as I know, all of the decent and popular backup programs can do this.

On a typical day, after I have deleted, renamed, moved, and added my normal amount of files, FreeFileSync can make the new backup in roughly one minute---the backup drive is then once again an exact replica of the source drive, reflecting the changes I've made on the source drive since the last backup.

You state: "When any given folder on the original and backup is compared, the program can't tell the difference between a file that is missing on main computer because I wanted to delete it permanently and one that was missing because I moved it to another folder or renamed it - correct?".

Not correct. If dog.jpg is deleted from the main computer, it would also be deleted from the target the next time the backup is run. If dog.jpg is moved to another folder on the main computer, it will be deleted from the original folder on the target and added to the new folder on the target the next time the backup is run. If dog.jpg is renamed to doggy.jpg on the main computer, then dog.jpg will be deleted from the target and doggy.jpg will be added to the target the next time the backup is run. ALL OF THIS WILL HAPPEN IF AND ONLY IF YOU USE MIRRORING.

Am I misunderstanding something about your situation?

catalogue · 17 Oct 2015

The catch is, my external has a hundred Gb or so of files that I keep only there, and not on the computer.

I think this messes up the whole mirroring thing that you're talking about, because when the program sees a file on the external that's not on the computer, it woudln't know if that's because I want it that way or because I want it deleted forever from both drives. I think I didn't specify this in my post.

ignatzatsonic · 17 Oct 2015

catalogue said:

The catch is, my external has a hundred Gb or so of files that I keep only there, and not on the computer.

I think I didn't specify this in my post.

I know you didn't.

For some reason, you can't keep this "hundred Gb or so of files" on the source drive and have them be part of your backup to the target drive, just like any other file. So be it. That's your choice.

You might be able to get around it by directing the backups to a folder on the target drive.

Assume the external is F.

Keep the 100 GB of stuff in the F:\yadayada folder.

Make a folder on F called yourmama.

Tell Synctoy to backup to F:\yourmama, not to F.

That might work. I've never experimented.

Golden · 18 Oct 2015

Ignatz last suggestion will work perfectly - backup to a different folder.

Victor S · 18 Oct 2015

It works. That's how I do it. And I only have to back up one folder. I call mine D-Backups. Everything I back up is under that folder.

McSeven · 18 Oct 2015

Goodsync is another great backup tool, been using it for years. Not free though.

https://www.goodsync.com/php/pums/rf...odsync&lang=en

catalogue · 20 Oct 2015

ignatzatsonic said:

You might be able to get around it by directing the backups to a folder on the target drive.

But then my photos would be spread out between two different directories, which causes all kinds of problems when I go to search for something. E.g. if I search for photos with a certain word in the filename I have to run the search in two separate folders one after the other and can't see the results all together; or else run the search on the *entire* drive and get results that aren't photos (which can be hundreds if the word I'm searching for is used, let's say, in software icons - then I could get hundreds of results that are jpg images but aren't my own photos).

Ultimately this might not be any less hassle than the manual brute-force backup I've been doing.

ignatzatsonic · 20 Oct 2015

What is your search tool?

You ought to try "Everything" from voidtools.com.

It can be configured to include or exclude certain drives, folders, or sub-folders.

You search by text string within the file name. A search for "bmp frank si" would find all pictures of frank sinatra or frank simon that had "frank si" within the file name and had a bmp extension, but not jpegs or tiffs or mp3s.

Results are shown in a single scrollable window.

Results can then be sorted by path, extension, size, type, date modified, creation time, last access time, or attributes.

Lightning fast--results displayed about as fast as you can enter the search term.

It's free.

It's useless for anything other than file names.

Victor S · 20 Oct 2015

You have added only ONE unique level of file structure to the hierarchy.
Whatever its name (Backup-Files for example) search on that.
And use a good search tool. I'm using Power Desk Pro's file search, but don't Power Desk Pro.
Find one that works for you.