Roy Schestowitz

Automatic Linux Backup

Foreword

This page presents an automated way of backing up all files, Web sites, and mail on a remote server or media. This can be done automatically by using a cron, for example. This page does not explain about crontabs, but one needs to ensure all paths are accessible to a cron. Alternatively, the code below can be aggregated to form a script that is invoked manually.

1. Backing up Files and Documents

Always make sure that new files are placed in a temporary directory. Once they have been backed up, they can be moved to the destined directory. It is as safe as it gets.

Let us say that the temporary directory is:

/home/[user]/Transfer_Archives/Files

Put the important files there and treat is as your main workspace.

2. Backing up Web sites

First of all, you will need to download all the important pages from your site/s. Here is an example:

cd /home/[user]/Transfer_Archives/Sites/Site1/
wget -l0 -t1 -N -np -erobots=off http://site1.com/index.php
wget -l0 -t1 -N -np -erobots=off http://site1.com/about.php
wget -l0 -t1 -N -np -erobots=off http://site1.com/archives.html

cd /home/[user]/Transfer_Archives/Sites/Site2/
wget -l0 -t1 -N -np -erobots=off http://www.site2.com/index.php
wget -l0 -t1 -N -np -erobots=off http://www.site2.com/frontpage.htm

This obtains an up-to-date local copy of the pages.

3. Compressing and Timestamping

Having got a collection of new or recent documents as well as Web pages, they need to be packaged together and filed with a datestamp.

tar czvf /home/[user]/Transfer_Archives/www-`date +%Y-%m-%d`.tar.gz /home/[user]/Transfer_Archives/Sites
tar czvf /home/[user]/Transfer_Archives/bu-`date +%Y-%m-%d`.tar.gz /home/[user]/Transfer_Archives/Files

4. Copying to a Remote Destination

cp -rf /home/[user]/Transfer_Archives/*.gz /home/[server]/Transfer_Archives/

In this particular case, [server] is the location of a mounted server. You might prefer burning a CD or copying to a second hard-drive.

5. Copying Settings to a Remote Destination

Finally, one can copy many Linux-specific settings and mail, just to ensure full and quick recovery shall disaster ever strike. The following is somewhat KDE-specific:

BACKUP_DIRS=/home/[user]/Mail/inbox
BACKUP_DIRS_OLD=/home/[user]/Mail/Old
BACKUP_FILE=/home/[user]/.kde/share/config/kmailrc
BACKUP_BOOK=/home/[user]/.kde/share/apps/konqueror/bookmarks.xml
BACKUP_BOOK2=/home/[user]/.mozilla/firefox/o5jcmbap.default/bookmarks.html

rm /home/[server]/backup/mail.tar.gz
rm /home/[server]/backup/mail_old.tar.gz
rm /home/[server]/backup/kmailrc.tar.gz
rm /home/[server]/backup/bookmark.tar.gz
rm /home/[server]/backup/bookmarkm.tar.gz

tar -cvzf /home/[server]/backup/mail.tar.gz $BACKUP_DIRS
tar -cvzf /home/[server]/backup/mail_old.tar.gz $BACKUP_DIRS_OLD
tar -cvzf /home/[server]/backup/kmailrc.tar.gz $BACKUP_FILE
tar -cvzf /home/[server]/backup/bookmark.tar.gz $BACKUP_BOOK
tar -cvzf /home/[server]/backup/bookmarkm.tar.gz $BACKUP_BOOK2

Possible Optimisations

The method above has a couple of imperfections:

There is no guarantee that failure in copying would be harmless
Since cp essentially merges files and skips files with identical timestamps where they exist already, there may be redundancies

I have devised a method to overcome these problems. Following the idea of commit/rollback in databases, one can keep two backup 'slots', each of which contains a back-up. On Sundays, Tuesdays and Thursdays, for example, all backup-ups will be put in slot 1. If successful, slot 2 will be purged. On the remaining days of the week, back-ups will be copied to slot 2 and once completed, slot 1 can be discarded. The main drawback, however, is bandwidth. If it is an overnight job, bandwidth may not be a significant factor.

5. Final Word

Run the scripts above after they have been tailored to suit your personal machine. Do so daily or weekly depending on bandwidth, volume of files, space available on server, etc.

G O O D L U C K

This page was last modified on March 26th, 2005

Maintained by Roy Schestowitz