Backup software that splits large files for incremental backup

[leuce]

G'day everyone

Does anyone know of a simple backup program that is capable of backing up large files incrementally by splitting the file and backing up only changed parts of the file? The files I'm talking about are files that get altered by appending stuff at the end of the file, e.g. log files or mailbox files, so it's generally possible to split a new version of the file at the exact point where the old version stopped, purely by file size.

Specifically, I want to back up Thunderbird and SeaMonkey mailboxes without having to create additional subfolders within those programs.

I understand that if something had changed near the beginning of the file, then the whole file will have to be backed up, unless the backup program is capable of some very clever searching, but mostly the files will change only by having data appended to it.

Thanks
Samuel

 

[fireberd]

You may want to rethink incremental backups. Where I worked, until I retired, we first used incremental backups for our servers. That seemed like a good approach - make an initial full backup on one day of the week and then only incrementals the rest of the week. That allowed smaller backups however when it came time to restore it was a nightmare making sure the incrementals were installed in the proper order, and it took much longer to restore. A decision was made to only do full backups every night and although the backup was bigger if a restore was needed there was no confusion on order of reinstall and it took less time.

I do the same thing with my own PC backups. I use Acronis True Image, which has incremental backup ability, but I only do full hard drive backups, to a separate hard drive.

 

[leuce]

When it came time to restore it was a nightmare making sure the incrementals were installed in the proper order, and it took much longer to restore.

Yes, but that is what backup software is there for. And yes, the restore would take somewhat longer, but that's the trade-off. In fact I don't even understand why restoring from an incremental backup should take any longer than restoring from a full backup, if you back up separate files as separate files.

There are two reasons I want to split large files. The first reason is bandwidth -- large files take more time to backup and they consume more bandwidth, neither of which is a a free, unlimited resource. The second reason is space -- some incremental backup systems keep old versions of files (or rather: some do not), so if you're backing up a large file, you get lots of copies of very large files, for no particular useful reason.

And particularly with files like log files or MBOX files, any changes to the files are almost always simply added to the end of the file, which means that splitting the file at precisly the right spot can't be simpler.

Sure, merging the files would take time, but backups are there for crisis management, not for synchronisation. Synchronisation is something else.

 

[Victor S]

Google "byte-level incremental backup"
It's probably considered to be an "enterprise" solution - which means it's costly.
You can decide if it's cost-justified.

 

[fireberd]

We backed up our file servers to tape. With an incremental backup, several tapes had to be loaded when a restore was required. With a full backup only one tape had to be loaded.

I'm fully aware of why there are backups. My job for 23 years until I retired was a regional (multi state) LAN/WAN Network and Hardware Help Desk Manager for a US Federal Goverment Agency. I was responsible for telecommunications and computers, for the entire area from west of the Mississippi to the Rockies (and from North Dakota to Texas) for all the agencies field offices. I had network help desk techs, modem techs and contract Novell CNE's working for me. I was also Novell CNE certified and along with the site manager I was also the site engineer. Prior to this job I worked as a Telemetry Processor Programmer at NASA.

 

[whs]

Why don't you backup to an external disk. Then you don't have the bandwith program. And never incrementals - they can be a nightmare. If any, use differentials.

Put your files into a seperate partition and use an imaging program. Most pro versions can do differentials - although I would always prefer full images that free imaging programs like free Macrium can do.