Proxmox configuration and backup

[Marc Ballat]

I am slowly recovering from an OVH dedicated server that didn't boot any more. I took the opportunity to upgrade from 4 to 5 and from dir to lvm-thin storage but it was a lot of pain and down-time. Fortunately and although I have a couple of production-grade apps running on it, it is only for my small own business and I have workarounds for most of the tasks.

As always, it is after a minor heart attack that one decides to quit smoking (major ones kill so there is no recovery in that case). So do I decide to improve the way I backup my Proxmox server, VMs and containers

In order to be able to backup to OVH's FTP server directly, I use curlftpfs and add the following line to /etc/fstab :

curlftpfs#ftpback-rbx2-97.ovh.net/              /mnt/ftpback    fuse allow_other,nonempty,uid=0,gid=1001,umask=0007,auto 0 0

I have come across patches that make it possible to make differential backups here https://ayufan.eu/projects/proxmox-ve-differential-backups/. Does anyone have experience with them ? I am planning to take daily differential backups together with weekly full backups. Is there a way built into Proxmox to apply a retention policy (like keep 12 last months, 4 last weeks, 7 last days) ?

Last but not least, the configuration of Proxmox itself is quite time consuming to restore (i.e. rebuild by hand). How can I better keep track of the configurations done with files like /etc/pve/storage.cfg, /etc/network/interfaces (that one was a real pain to rebuild) and /etc/fstab ?

Thanks in advance.

Marc

 

[Faris Raouf]

Although I have no experience of them myself, from what I have read here on the forum, these differential backups are a bad idea. Stick to full backups.

Once you have a backup, how about copying them to something like Amazon S3? Amazon allows you to create a policy on a bucket that will move a file after X days to Glacier. Or you can go the whole hog and move from normal S3 to infrequent access storage and then the Glacier. It isn't quite what you are looking for, but it is sort of similar in a way.

Yet another option is to use something seriously smart, like Jungledisk. Jungledisk does deduplication as it backs up. I have failed to get it working in a Container in Procmox so far, but on my old setup (Virtuozzo), where I had it running on the node itself, it would look at my huge backup files of various VZ Containers, and only backup what was absolutly needed. It was almost magic, because these backups were compressed Acronis .tib files. How it managed to deduplicate is beyond me. I can only imagine that large blocks of these backup files were all very similar (they were all Centos containers).

I do not know how well this would work on .lzo backups.

But I do know it was saving me huge amounts of disk space for my backups. I'm currently storing 2.5TB using bare .lzo copied to S3. With Jungledisk I was using about 500Mb.

 

[slanbuas]

An option I used in the past was to backup VM and containers uncompressed and then use borg backup to backup the data. Since it can deduplicate/compress and can have retention you can get pretty good results. https://www.borgbackup.org

I remember that the deduplication wasn't optimal because of the format proxmox uses but still had great results.

 

[RolandK]

we're using borgbackup directly on proxmox host, backing up from zfs snapshot with good results so far (still in pre-production/testing phase @dingfabrik fablab cologne,as we bounce ESXi - but it looks promising)

see for special tuning : https://mail.python.org/pipermail/borgbackup/2019q1/001280.html