Hi,
First of all thank you Proxmox Team for such a great solution. I just started using Proxmox and am very impressed.
I also like to contribute my backup strategy which I figured after some time of consideration.
Because of its simplicity and suspected long-term feasibility, I use vzdump to create dumps of all VZs (which are stored on LVs) to a local folder.
Afterwards, duplicity is used to stream incremental delta backups to an ftp server (could be a lot of other storage backends as well, e.g. Amazon S3, see the duplicity documentation). I believe duplicity is the best complement to vzdump so far because of the following reasons:
- suspected long-term feasibility: Used as a backend for the default Ubuntu backup solution dejadup
- amazingly idiot proof for a Linux program (I am very impressed about this), i.e. it tries to intelligently handle orphaned configuration files of deleted backups
- very efficient incremental backups (only about 1MB compressed delta size between 2 VZ snapshots. Delta was due to logfiles etc.). Be sure to have the compression with vzdump disabled as otherwise the duplicity incremental backup would work very inefficiently (I guess, not tested)
- just transfers deltas over the wire, so incremental backups are rather fast (and small)
- built-in encryption based on PGP
I do not use duply in addition because it adds more complexity here instead of making things a lot easier.
To get a basic setup running, try the following:
Code:
> apt-get install mmv duplicity ncftp
create the following script /usr/bin/backupVZs.sh:
Code:
#!/bin/sh
cd /mnt/localbackup/vzDumps/
## rename the VZ images (if any) to a common name scheme (so that consecutive images get the same name) overwriting the previous image
## This is necessary for the incremental backup with duplicity, it removes all timestamps from the filenames. Unfortunately, it seems there is currently no way to tell vzdump to always use the same name
mmv -d "vzdump-qemu-*-*.*" "vzdump-qemu-#1.#3"
mmv -d "vzdump-openvz-*-*.*" "vzdump-openvz-#1.#3"
## transfer previous VZ images to ftp (if any)
PASSPHRASE=arbitraryPasswordForEncryptingTheArchieves FTP_PASSWORD=passwordForFTPServer duplicity /mnt/localbackup/vzDumps/ ftp://ftpuser@backupserver.org
## create new VZ images
## they are transferred just before the next backup
vzdump --dumpdir /mnt/localbackup/vzDumps/ --snapshot --all --bwlimit 0
Code:
Make the sript executable
>chmod u+x /usr/bin/backupVZs.sh
You can now use the cron deamon to do backups on a regular basis.
Add this line to make the backup every day at 00:00
Code:
0 0 * * * /usr/bin/backupVZs.sh
Initially, it will create a new snapshot backup of all your VZs to the local backup storage (here: /mnt/localbackup/vzDumps/).
On the next invokation, it will normalize the snapshot file names (so that incremental backups will work) and transfer the snapshots to the ftp server
After that, new snapshots are created. If you have limited space on the local backup storage, you could delete the previous snapshots (which are already transferred to the ftp server) before this step.
You might want to change the order of the entire procedure if you prefer to directly transfer the most recent version to the ftp server. There are also a lot of other options that you might want to influence, so take a look at the duplicity docs.
Of course, there is always a room for improvement and I'd like to make some suggestions in case some of the Proxmox or Duplicity Gurus are listening
- I don't know how complicated this can get but an in memory backup directly to the remote storage (without having to store the images first locally) would be fantastic. I guess this requires a much deeper integration of vzDump and Duplicity, but maybe I pipe would be enough? I.e. instead of taring, vzdump could pipe the contents to duplicity (currently unsupported i believe) which will do the rest...
- Second thing is that this should be controllable from the Web gui, so no shell crontab fiddling any more...
- Restoration of individual files from the images is a bit of pain right now. A nice web based version chooser (which image version to restore) would be nice too...
Cheers,
fatzopilot
