Backup system for little installation (on zfs)

[fcorbelli]

I am developing a small project to archive backups from zfs, for now only in the simplest possible version

That is, a single local, with virtual machines whose disks are all contained in /var/lib/vz
In short, a small installation

As you can understand, this is a backup of a small OVH server I am taking care of, and I share the experiences I am gradually accumulating

The aspects I find most interesting are the ease of encryption, the occupation of a modest amount of space (deduplication + compression), compatibility with cloud copy systems with minimal transfer (rsync --append)

Obviously, the same (or even better) result can be achieved by the 'concatenated' use of various tools

Ease of use is one of the most important requirements: as mentioned, it is NOT designed to replace Proxmox backup, but to take advantage of zfs snapshots

This is an opensosurce C++ software, so it requires either a gcc compiler or (for the really brave!) directly using a binary compiled by me

I am not even sure that backups are able to be restored (!), it is just at an early stage of development

I would be very happy if some users could help me by testing the utility and giving back suggestions for error correction etc

I realise that using binary code from an "unknown random user" is not the safest thing in the world (!) so I suggest to start from the source

This is actually a 'nightly' build, here is the complete project (without proxmox support)
https://sourceforge.net/projects/zpaqfranz/
https://github.com/fcorbelli/zpaqfranz

wget http://www.francocorbelli.it/Makefile
wget http://www.francocorbelli.it/zpaqfranz.cpp -O zpaqfranz.cpp
make install clean

In this example the zfs datasets are... really... basic...




After that, you can run something like ... (-key pippo means "with password pippo")

Don't be afraid of "-force" and "-kill"
-force means PROCEDE ANYWAY EVEN IF A SNAPSHOT ALREADY EXISTS
-kill meas KILL THE SNAPSHOT AFTER THE RUN

zpaqfranz zfsproxmox /backup/backup_200.zpaq 200 -force -kill -key pippo
zpaqfranz zfsproxmox /backup/backup_200_300.zpaq 200 300 -force -kill
zpaqfranz zfsproxmox /backup/backup_all.zpaq -all -force -kill -key pippo

Once you have made a backup, you can repeat it, as many times as you like, to store the various versions (in short, snapshots) within the single .zpaq archive.

Data extraction (from the zpaq archive) is possible both - of course - with my zpaqfranz and directly with zpaq, which is present in version 7.15 that can be installed 'brutally' with

apt-get install zpaq

So, to recapitulate, extraction is guaranteed at all times (a fundamental element in the use of an 'esoteric' new tool)
https://en.wikipedia.org/wiki/ZPAQ

Thanks for all feedback, even negative ones!

 

[fcorbelli]

What exacly runs "BTS"?

• Analyze the zfs list to find (from the mountpoint of /var/lib/vz) the dataset
• Search for the .conf (s) files in /etc/pve/qemu-server/ (quick-and-dirty)
• Destroy a "francoproxmox" snapshot (can be changed via -to, but it is good enough, unless your name is Franco just like mine
• Make a "francoproxmox" snapshot (a fresh one) on the dataset
• Archive the virtual drives (better, the folders with inside the drives) + the .conf file(s) inside the .zpaq archive
• Destroy the francoproxmox snapshot (if -kill), or retain (to make other kind of backups, like rsync for example)

That's all, nothing "magic", you can do with (for example) 7z or tar or whatever

On the next run the sequence is repeated,
BUT
inside the .zpaq archive every single version (snapshot) is stored
THEN
you can (MAYBE! never tested so far! restore later to the X-taken snapshot

 

[fcorbelli]

Last step: the remote "cloud" backup, with something like that (of course just a snippet)
Please look at "--append", this is the "key"

PORTA=22
UTENTE=franco
SERVER=my.backup.whatever
CHIAVE=/backup/my_RSA_KEY

if ping -q -c 1 -W 1 $SERVER >/dev/null; then
/bin/date +"%R ---------- PING => upload"
/usr/bin/rsync  -vv -I --append --omit-dir-times --no-owner --no-perms --partial --progress -e "/usr/bin/ssh -p $PORTA -i $CHIAVE "  -rlt  --delete "/backup/" "$UTENTE@$SERVER:/home/franco/proxmox/"
/bin/date +"%R ---------- END UPLOAD"
else
/bin/date +"%R CANNOT PING!"
fi