====== Multiplicity ======

Multiplicity is a very basic frontend to [[http://www.nongnu.org/rdiff-backup/|rdiff-backup]]. Its goal is to provide easy management of multiple independant backup tasks.

===== Overview =====

''rdiff-backup'' uses ''rsync'' (incremental file tree copies to save bandwidth), hard links (to save backup space), ''ssh'' (to transmit data) and a very handy incremental machinery (to allow us to go back in time through our backups).

''multiplicity'' is a script aimed at easing the management of backup jobs. It is a basic frontend to ''rdiff-backup''. It consists of the main scipt and a directory filled with task files. Each task file defines a backup source, a backup target and various other parameters.

===== Task files =====

A task file must be in the configuration directory (may be set on the command line and defaults to ''/etc/multiplicity''). It should be named with a ''.task'' suffix (read dot task). The following configuration values are accepted in a task file :
  * ''run_this'' : This globally enables or disables the task file. Any value other than ''yes'' disables the task file.
  * ''max_age'' : delete files deleted //more than this value// ago and file history //older than this value//. This parameter is directly passed to ''rdiff-backup''.
  * ''source_root'' : the root directory of the files you want to backup.
  * ''target_root'' : the root directory where you want to backup your files.
  * ''include_file'' : a full path (or relative to the configuration directory) to a file that lists what files to include in the backup. As the default is to backup all files, this is only usefull if the files you list here would have been excluded regarding the ''exclude_file'' configuration. See next dot.
  * ''exclude_file'' : files to exclude from the backup.
  * ''pre'' : execute this in the ''bash'' shell before executing the backup task. This must fit on one line.
  * ''post'' : execute this after the backup task. This must fit on one line.

Example task file ''/etc/multiplicity/example.task'' :
  run_this        no
  max_age         185D
  source_root     /
  target_root     /backup
  exclude_file    local.exclude
  pre             mount -o remount,rw /backup
  post            mount -o remount,ro /backup

Carefull, what separates configuration keys from configuration values is a real tabulation ''\t''. I should probably fix this, one day.

Example exclude file ''/etc/multiplicity/local.exclude'' :
  /var
  /proc
  /bin
  *.log

===== Global pre and post scripts =====

There is also the possibility of running global pre and post scripts, i.e scripts that run before and after the task batch. Those are to put in the configuration directory under the names of ''pre.sh'' and ''post.sh''.

Example ''pre.sh'' :
  #!/bin/bash
  # This script is run before the backup task batch begins.
  # Feel free to customize it
  
  # Dump mysql data to a file
  mysqldump --user root --password=pass --all-databases > /var/backups/mysqldata base.dump

===== Usage =====

''multiplicity'' accepts the following command-line options :

  * ''-c__confdir__'' : Specify ''confdir'' as the configuration directory, i. e. where to look for task files.
  * ''-v'' : Be verbose. Default is really quiet.
  * ''-h'' : Display a (very) short help summary.

===== Download =====

You can browse [[http://sousmonlit.dyndns.org/~niol/darcsweb/darcsweb.cgi?r=multiplicity;a=summary|multiplicity's source]] online.

You can get the source using ''darcs'':

  darcs get http://sousmonlit.dyndns.org/~niol/repository/multiplicity

===== Installation (short) =====

''multiplicity'' may run in any directory, but it was made to be run in the following way :

  * Create the directory ''/etc/multiplicity'' and create ''task'' files. You can re-use the example from the distribution.
  * Drop the ''multiplicity'' script in a ''cron'' directory if applicable, for example on Debian, I dropped it in ''/etc/cron.daily'' to make it run once a day. After a couple of days, you may want to put ''VERBOSE'' to a zero value to stop getting cron report emails.

===== Installation (my setup) =====

This section describes my setup. The goal is to backup from ''ripley'' to ''backup1'' and ''backup2''.

==== Password-less SSH logins ====

The first thing to do is to setup password-less SSH logins for the ''root'' user to the hosts you want to backup to (or the hosts you want to backup from, but I'll describe here how to use remote backup hosts to backup //to//). First create a key pair for this purpose.

  # cd
  # mkdir .ssh
  # chmod 700 .ssh
  # ssh-keygen -t rsa -f ~/.ssh/id_rsa
  (no passphrase)

If, like me, you have aliases for your hosts and special ports, you may want to use a ''.ssh/config'' file :

  Host backup1
  User backupuser
  Port 400
  HostName backupfool.foo1.org
  
  Host backup2
  User backupuser
  Port 400
  HostName backupguy.foo2.org

Now, copy your IDs to the remote hosts (repeat for ''backup2'') :

  # ssh-copy-id -i ~/.ssh/id_rsa.pub backupuser@backup1
  Password:

Now you should be able to login with the local ''root'' user on ''localhost'':

  # ssh backup1
  (no password prompted)
  backupuser@backupfool.foo1.org $

==== Multiplicity configuration ====

So, here is the plan : I want to backup several selected directories to a dedicated ''/backup'' partition and I want to backup only a part of those selected directories to the remote hosts, especially avoiding to send to the remote hosts my passwords and my emails.

I run a mySQL server on ''ripley''. Thus, I use ''/etc/multiplicity/pre.sh'' to dump the database contents to a file :

  #!/bin/bash
  mysqldump --user root --password=unbreakable_password --all-databases > /var/backups/mysqldatabase.dump

Then I described ''/etc/multiplicity/local.include'' which is a list of the things on ''ripley'' that I want to backup :

  /etc
  - **tmp
  /home/niol
  /root
  - /var/fileserv/enac/album**tn
  /var/fileserv/enac/album/**jpg
  /var/backups
  /var/spool/minimalist
  /var/www
  - **

Basically, I want to backup ''/etc'', no files that end with ''tmp'', my personal directory, stuff on my ftp server except thumbnails directories, ''/var/backups'', my mailing list manager config, my web directory and nothing else. Check the ''backup-manager'' manual page for more information on file inclusions and exclusions (globbing filelists).

But when backing up to a remot host, I want to omit some files (the contents of the ftp server which is too big and the ones that contain clear text passwords) which I described in ''/etc/multiplicity/remote.exclude'':

  /var/fileserv/
  **id_rsa
  **id_dsa
  **identity
  **gpg*
  **.bash_history
  **.fetchmailrc
  **.gnubiffrc
  /etc/ssh/ssh_host*key
  /etc/multiplicity/pre.sh
  /home/niol/fromannalee/home/niol/.gaim
  /home/niol/fromannalee/home/niol/.mozilla
  /var/www/sousmonlit.dyndns.org/sql.conf.php

(I really don't like clear-text-stored passwords...)

This was the most difficult part bescause you need to know what is on your computer. But now, the only remaining thing is to describe multiplicity tasks.

''/etc/multiplicity/local.task'' (the local backup):

  run_this        yes
  max_age         185D
  source_root     /
  target_root     /backup/local
  include_file    local.include
  pre             mount -o remount,rw /backup
  post            mount -o remount,ro /backup

''/etc/multiplicity/backup1.task'' (the order of ''exclude_file'' and ''include_file'' is important):

  run_this        yes
  max_age         10D
  source_root     /
  target_root     backup1::/home/backupuser/fromripley/
  exclude_file    remote.exclude
  include_file    local.include

''/etc/multiplicity/backup2.task'' (this is the second friend I send my backups to, just in case):

  run_this        yes
  max_age         10D
  source_root     /
  target_root     backup2::/home/backupuser/fromripley/
  exclude_file    remote.exclude
  include_file    local.include

This should be self explainatory and this should show what is ''multiplicity'' about.

==== Running the script ====

The first run should be a bit long :

  # /etc/cron.daily/multiplicity -v

From now on, you can check on the remote hosts which files have been backed up :

  # ssh backup1
  backupuser@backupfool.foo1.org $ ls fromripley
  etc  home  rdiff-backup-data  root  var

You also may check what increments are available :

  backupuser@backupfool.foo1.org $ rdiff-backup -l fromripley
  Warning: ownership cannot be changed on filesystem at /home/backupuser/fromripley/rdiff-backup-data
  -----------------------------------------------------------------
  Detected abilities for archive (read/write) file system:
    Characters needing quoting                   ''
    Ownership changing                           Off
    Hard linking                                 On
    fsync() directories                          On
    Directory inc permissions                    On
    Access control lists                         Off
    Extended attributes                          Off
    Mac OS X style resource forks                Off
    Mac OS X Finder information                  Off
  -----------------------------------------------------------------
  Found 9 increments:
      increments.2005-12-27T06:30:54+01:00.dir   Tue Dec 27 06:30:54 2005
      increments.2005-12-28T06:31:53+01:00.dir   Wed Dec 28 06:31:53 2005
      increments.2005-12-29T06:31:03+01:00.dir   Thu Dec 29 06:31:03 2005
      increments.2005-12-30T06:32:27+01:00.dir   Fri Dec 30 06:32:27 2005
      increments.2005-12-31T06:33:26+01:00.dir   Sat Dec 31 06:33:26 2005
      increments.2006-01-01T06:32:10+01:00.dir   Sun Jan  1 06:32:10 2006
      increments.2006-01-02T06:31:25+01:00.dir   Mon Jan  2 06:31:25 2006
      increments.2006-01-03T06:30:41+01:00.dir   Tue Jan  3 06:30:41 2006
      increments.2006-01-04T06:31:11+01:00.dir   Wed Jan  4 06:31:11 2006
  Current mirror: Thu Jan  5 06:31:08 2006

Check the ''rdiff-backup'' [[http://www.nongnu.org/rdiff-backup/examples.html#restore|beginners guide]] to know how to recover an old file...