This post originated from an RSS feed registered with Web Buzz by Stuart Langridge. Original Post: Backups on a Bytemark VM Feed Title: as days pass by Feed URL: http://feeds.feedburner.com/kryogenix Feed Description: scratched tallies on the prison wall Latest Web Buzz Posts Latest Web Buzz Posts by Stuart Langridge Latest Posts From as days pass by

The server that this site runs on is a Bytemark Hosting virtual machine. Those of you who have been reading for a while will remember that we had a serious fuckup a couple of months ago and the site went away. And we didn’t have any backups. Fortunately, we were able to retrieve an old version of the sites hosted here, and reconstructed the data. But that’s not happening again, so I’ve set up some backups. Bytemark give you some free backup space to back up your VM. The following steps should give you full nightly backups of everything you want to back up.

mkdir /backup/.config
mkdir /backup/rsync
mkdir /backup/rsync/files
mkdir /backup/rsync/db
nano /backup/.config/rsync

VMNAME-backup:PASSWORD:::1:rsync

mkdir /usr/local/backups
echo "RSYNC_PASSWORD_FROM_ABOVE" > /usr/local/backups/backup-password
chmod og-rwx /usr/local/backups/backup-password

rsync -a \
--password-file=/usr/local/backups/password \
/etc/motd \
VMNAME-backup@VMNAME.backup.vm.bytemark.co.uk::VMNAME-backup/files/test/

Now to create two backup scripts, one to do your MySQL databases and one to do your files. If you don’t use MySQL, skip the DB one (or replace it with one that does your own choice of DB.)

#!/bin/bash

# Dump all databases to a file, and rsync that file into the backup space,
# inside a directory named for data and time.

OUTPUT_FILE=/usr/local/backups/dbdump
NEW_BACKUP_DIR=`date +%Y%m%d@%H%M%S`

mysqldump --all-databases --user=root --password=_yourMySQLRootPassword \
   --opt | bzip2 > $OUTPUT_FILE

rsync -a \
--password-file=/usr/local/backups/backup-password \
 $OUTPUT_FILE \
 <em>vmname</em>-backup@_vmname_.backup.vm.bytemark.co.uk::_vmname_-backup/db/$NEW_BACKUP_DIR/

logger -t Backups Backup of all databases done to $NEW_BACKUP_DIR

#!/bin/bash
# Backs up files with rsync

NEW_BACKUP_DIR=`date +%Y%m%d@%H%M%S`
FILES_TO_BACKUP=/usr/local/backups/files-to-include
EXCLUDE_FILES=/usr/local/backups/files-to-exclude
LAST_BACKUP_FILE=/usr/local/backups/last-file-backup

# Work out what the <strong>last</strong> backup was, to use as a base for this one
if [ -f $LAST_BACKUP_FILE ]; then
  read LAST_BACKUP < $LAST_BACKUP_FILE
else
  LAST_BACKUP=nonexistent
fi

echo $NEW_BACKUP_DIR > $LAST_BACKUP_FILE

rsync -a --delete \
  --password-file=/usr/local/backups/backup-password \
  --include-from=$FILES_TO_BACKUP \
  --link-dest=../$LAST_BACKUP \
  / \
 VMNAME-backup@VMNAME.backup.vm.bytemark.co.uk::VMNAME-backup/files/$NEW_BACKUP_DIR/

logger -t Backups Backup of files done to $NEW_BACKUP_DIR

# List of directories to back up
# If you want to back up directory /a/b, then you need the following lines:
# /a         (since otherwise we never get as far as b, because a is excluded)
# /a/b       (the directory we want to back up)
# - /a/*/    (don't back up any other directories in /a)
# - /a/*     (don't back up any files in /a either)
# Yes, this is completely shit. Rsync's fault.

/var/
/var/www/
- /var/*/
- /var/*
/home/
/etc/
/usr/
/usr/local/
- /usr/*/
- /usr/*
/usr/local/backups/
- /usr/local/*/
- /usr/local/*

# Directories to explicitly exclude. Note that you don't need all
# that parent directory shit here; just put in dirs that you don't
# want to back up for whatever reason.

- /var/www/jonobacon.org/mysqlbackups/
- /var/www/jonobacon.org/mysqlbackups.old/

# Don't fuck about with the stuff here; it ensures that logs directories
# aren't backed up, and neither is anything else in / (otherwise we'll
# back up the whole machine or something equally dreadful).

- **/*.mp3
- **/logs/
- /*/
- /*

This is actually a super clever backup method. All the magic is done by rsync’s—link-dest option. What that does is, every time you tell it to back up, it creates a tree of hardlinks in the backup space from the last backup. Then it rsyncs any changed files over the top. This means that every single backup is a full backup, but it only takes up the space of an incremental backup. This only works because hardlinks exist. I love Linux, I really do.

Anyway, those two files should do the job of backing up everything. Stick two lines in your crontab to kick them off.

# ########### Backups ###############
# mysql
30 5 * * * root /usr/local/backups/backup-databases.sh

# files (4.40am)
45 5 * * * root /usr/local/backups/backup-files.sh

and it should all work.

Note that these instructions probably won’t work for you right off, but they will at least give you a starting point.