For some time, I have been looking for a good backup strategy for a linux system. Currently, my backup strategy was
(server1) –> daily rsync –> (backupsever)
(backupserver) –> daily tar.gz –> (archiveserver — keep multiple copies)
While this worked, my archive server was always overloaded and I really only had a ‘few’ copies of a file. backupserver had yesterday’s copy and archiveserver had copies for few days.
Then I came across this article – http://www.mikerubel.org/computers/rsync_snapshots/ . It uses a combination of rsync + hard links. A variation that may work for me, gleaned from the information on the page, is
(daily on backup server)
# Roll up the backups
mv backups.n backups.tmp
mv backups.n-1 backups.n
mv backups.n-2 backups.n-1
mv backups.n-3 backups.n-2
mv backups.n-4 backups.n-3
….
mv backups.1 backups.2
mv backups.0 backups.1
#Syncing the oldest backup (.n) to the latest
mv backups.tmp backups.0
cp -al backups.1/. backups.0 # This would hardlink files from .1 to .0, which were hard links to begin with
rsync -a –delete user@server1:source_directory/ backup.0/
(archiveserver – only one copy needed)
cd backup.0
tar czf /archive/backup.tgz .
scp ./archive/backup.tgz user@archiveserver:/archive
/bin/rm -f ./archive/backup.tgz
Ok – I found a way to use tar to create ‘incremental’ backups. They key is to use the ‘-g’ option (see http://mikesmithers.wordpress.com/2010/12/11/twiddling-with-tar-%E2%80%93-differential-backups-on-linux/)
cd backup.0
tar -cvzf /archive/backup.0.tgz -g /archive/backup.0.alist . # Level 0 backup since backup.0.alist file is not present
tar -cvzf /archive/backup.1.tgz -g /archive/backup.0.alist . # Level 1 backup only, only files NOT in backup.0.alist. Hmm …. does it backup changed files since backup.0 ? Need to read this.
(Update)Tar actually supports incremental and differential. See http://www.gnu.org/software/tar/manual/html_section/Incremental-Dumps.html
(Incremental)
(1st of the month)
tar –create –file=archive.0.tar –level=0 –listed-incremental=/var/tmp/archive.snar .
(other days)
tar –create –file=archive.dd.tar –listed-incremental=/var/tmp/archive.snar .
(Differential)
(1st of the month)
tar –create –file=archive.0.tar –level=0 –listed-incremental=/var/tmp/archive.snar .
(other days)
cp /var/tmp/archive.snar /var/tmp/archive.0.snar
tar –create –file=archive.dd.tar –listed-incremental=/var/tmp/archive.0.snar .
/bin/rm /var/tmp/archive.0.snar
.