Making backups snapshottable

[carl0s]

Hi guys.

I gather the changed block incremental backup stuff isn't quite production ready in QEMU yet, and not in Proxmox at all.

However, my NFS machine, the backup target, is running btrfs.

If only Proxmox had the option to always backup to the same filename instead of date-stamp filenames (and also I disable compression), then would this not give me space efficient multiple backups? btrfs snapshots would just capture the changed parts? What do you think?

 

[LnxBil]

Disabling the compression does not help. The binary layout changes, so you will accomplish nothing by per block deduplication or things like that. I tried all of it and digged into the vma stuff the Proxmox team did in their qemu development repository. The format is unfurtunately not deduplicatable. You can, however unpack the vma file and store the contents (raw disk image and config) on the btrfs itself and use the snapshots. I use a similar technique based on ZFS with every vm on its own dataset. Every time I need to restore a backup, I repack the extracted files back into a vma and put it on the NFS share for PVE to read.

 

[carl0s]

Disabling the compression does not help. The binary layout changes, so you will accomplish nothing by per block deduplication or things like that. I tried all of it and digged into the vma stuff the Proxmox team did in their qemu development repository. The format is unfurtunately not deduplicatable. You can, however unpack the vma file and store the contents (raw disk image and config) on the btrfs itself and use the snapshots. I use a similar technique based on ZFS with every vm on its own dataset. Every time I need to restore a backup, I repack the extracted files back into a vma and put it on the NFS share for PVE to read.

Ah that's a shame. I recently discovered that windows server backup within the guest os does this perfectly (using some hyper-v related vhdx and bleeding edge SMB3 stuff). It seems to just append new data to the same vhdx file on the samba share every time, which is super snapshot friendly it was however awkward to get windows server 2016 backup to write to a samba share at all (all sorts of new SMB3 stuff that's not fully complete in samba)

I will have a dig into what you suggested and try that out. Thanks for the tip.