Backing up VMs with very large storage drives- what's the best solution?

[Landrin201]

Hello! I run my home server via proxmox, and ran into a rather big snag last night that has me re-considering how I have everything set up. I need to explain how I have everything set up for any of this to make sense.

I have 2 vms in my setup, one runs HAOS and the other runs Debian. The debian VM gets 3 disks: one is the root drive, one is an SSD storage, and the third is a 20tb QEMU disk that lives on a 20tb ZFS raid.

Last night I broke the entire os with an upgrade from deb12 -> 13. It couldn't boot, and after about 4 hours of troubleshooting I decided to apply a backup. Problem: my backups all back up the ENTIRE vm, including the 20tb storage drive. This will end up taking at least 18 hours, just by estimating how long the first 5% has taken.

In the future, I'd like to avoid this. The problem I ran into had nothing to do with the storage drive, and could have been resolved by simply re-applying the root drive's image from Saturday night's backup.

I know that proxmox will allow me to select which drives to backup when I run a backup, but what about when I apply the backup? From a little googling I found a few threads that indicate that if I have drives A, B, and C, and I only backup A and B, when I restore that backup C will be wiped. Is that true? Because obviously I don't want that to happen, I don't want to lose the data stored on drive C, I just don't want to re-apply that data in a backup because it's 20tb large and doesn't need to be restored.

To be clear I still intend to back the 20tb drive up weekly, but if I could also do daily backups of ONLY the root drive and be able to apply those without effecting the storage drives that would be great.

If that isn't possible via proxmox, is there a solution that I could use in my debian vm to backup/restore just the root drive without touching the storage drives? All I really need is something to let me recover the root drive in case of a major upgrade failure...

 

[UdoB]

I have no solution for your A, B, C problem, just a hint regarding...

This will end up taking at least 18 hours

Did you notice that there is a "Live restore" checkbox in the Restore dialog?

The VM will start (nearly) immediately, with only restoring the currently, actively needed bits and bytes. Accessing any data requires the system to pull it from the backup, which might be slow, but it is transparent/invisible.

Of course to finish the restore takes at least as long as before, probably a little bit longer.

There are other drawbacks (it requires a PBS, no backups during this restore possible, if it crashes data "in-flight" is lost and some more) but in my own, limited experience it works really great.

Reference: https://pve.proxmox.com/pve-docs/chapter-vzdump.html#_live_restore

 

[Landrin201]

I'll keep that in mind for the future if I need to run full data restores, but I'd really rather avoid that where possible and just restore the root partition in case of update failure. Is there really not a way to do that?

 

[bl1mp]

Hi Landrin,
which solution/setup do you use?
Could you elaborate a bit on that? That will be helpful for useful advice.

Do you use the builtin pve capability for full backups? Do you use USB-Storage? What kind of disk image format do you use?

BR, Lucas

 

[Kingneutron]

Don't reckon I would ever give 20TB of disk space to a single VM. That's NAS-level storage, and needs to be redundant. 2-4TB of disk space for a single vm would likely be pushing my personal limits of "comfortable with it"

You might be better off putting that data at the host level on ZFS (+ mirror it, so you get self-healing scrubs) and either virtiofs-mounting or use a network share such as Samba for it. And make sure everything is on UPS power.

Thus you would have a small (appropriately sized) OS disk for the vm, easily backed up, and the large data set would be accessible from the host (and get host-based disk speed)

Edit: BTW, you have my sympathy. Seems like situations like this, we learn to separate OS + data the hard way...