dt-s3-backup icon indicating copy to clipboard operation
dt-s3-backup copied to clipboard

Bash script for automated backups with duplicity and Amazon's S3 online storage

Active version of script: https://github.com/zertrin/duplicity-backup

NOTE (May 6th, 2012): I am no longer actively developing this script.
zertrin has recently offered to take over for the time being. I have not, myself, tested his fork or the merges he's made but more development is always better than less. If you have pull requests or changes, use his branch. I will be quietly watching from the shadows. Good luck! -Damon

ABOUT THIS SCRIPT

This bash script was designed to automate and simplify the remote backup process of duplicity on Amazon S3. After your script is configured, you can easily backup, restore, verify and clean (either via cron or manually) your data without having to remember lots of different command options and passphrases.

Most importantly, you can easily backup the script and your gpg key in a convenient passphrase-encrypted file. This comes in in handy if/when your machine ever does go belly up.

Optionally, you can set up an email address where the log file will be sent, which is useful when the script is used via cron.

More information about this script avaiable at: http://damontimm.com/code/dt-s3-backup

Latest version of the code is available at: http://github.com/thornomad/dt-s3-backup

BEFORE YOU START

This script requires user configuration. Instructions are in the file itself and should be self-explanatory. Be sure to replace all the foobar values with your real ones. Almost every value needs to be configured in someway.

You can use multiple copies of the script with different settings for different backup scenarios. It is designed to run as a cron job and will log information to a text file (including remote file sizes, if you have s3cmd installed). Be sure to make the script executable (chmod +x) before you hit the gas.

REQUIREMENTS

  • duplicity
  • gpg
  • Amazon S3
  • s3cmd (optional)
  • mailx (optional)

COMMON USAGE EXAMPLES

  • View help: $ dt-s3-backup.sh

  • Run an incremental backup: $ dt-s3-backup.sh --backup

  • Force a one-off full backup: $ dt-s3-backup.sh --full

  • Restore your entire backup: $ dt-s3-backup.sh --restore You will be prompted for a restore directory

    $ dt-s3-backup.sh --restore /home/user/restore-folder You can also provide a restore folder on the command line.

  • Restore a specific file in the backup: $ dt-s3-backup.sh --restore-file You will be prompted for a file to restore to the current directory

    $ dt-s3-backup.sh --restore-file img/mom.jpg Restores the file img/mom.jpg to the current directory

    $ dt-s3-backup.sh --restore-file img/mom.jpg /home/user/i-love-mom.jpg Restores the file img/mom.jpg to /home/user/i-love-mom.jpg

  • List files in the remote archive $ dt-s3-backup.sh --list-current-files

  • Verify the backup $ dt-s3-backup.sh --verify

  • Backup the script and gpg key (for safekeeping) $ dt-s3-backup.sh --backup-script

TROUBLESHOOTING

This script attempts to simplify the task of running a duplicity command; if you are having any problems with the script the first step is to determine if the script is generating an incorrect command or if duplicity itself is causing your error.

To see exactly what is happening when you run dt-s3-backup, head to the bottom of the user configuration portion of the script and uncomment the ECHO=$(which echo) variable. This will stop the script from running and will, instead, output the generated command into your log file. You can then check to see if what is being generated is causing an error or if it is duplicity causing you woe.

NEXT VERSION WISH LIST

  • --restore-dir option
  • restore files from a specific time period
  • support multiple MTAs for the email alert feature

Thanks to Mario Santagiuliana and Razvan for their help.