Dirty Bitmap Backups are Wearing on Me

[whatishappening]

Is there an alternative way to configure backups that I'm unaware of? I have one very large data set at just over 9TB even with a 10 gig direct connection between PVE and PBS this takes AGES due to disk IO constraints then you have to wait on verification jobs, it takes days. Now throw in the regular kernel update reboots (I'm glad things are being patched) I'm constantly running this backup for multiple days each week. It's starting to make me anxious to reboot knowing it's going to chew on this backup for days each time.

I noticed there was a document talking about dirty bitmap persistance and was curious if there was a way to utilize this with proxmox?

https://qemu-project.gitlab.io/qemu/interop/bitmaps.html#bitmap-persistence

Or if I'm just doing this wrong somehow and need to make a change to my backup method.

 

[Impact]

Related issue: https://bugzilla.proxmox.com/show_bug.cgi?id=3233

 

[whatishappening]

Related issue: https://bugzilla.proxmox.com/show_bug.cgi?id=3233

Dang since 2021 eh? Guess this isn't getting implemented any time soon.

What other options do I have. I've read I could move the large data set out of a VM and into an LXC container and share it via NFS share so that I can use pxar for change tracking which (if I understand correctly) would persist through reboots. Does this check?

 

[Impact]

That could be an option but for NFS the CT needs to be privileged and you're involving the network which makes things more complicated/slow.

 

[whatishappening]

That could be an option but for NFS the CT needs to be privileged and you're involving the network which makes things more complicated/slow.

Ah good point, could switch to SMB to stay unprivileged. Data would stay all on host on a virtual bridge so wouldn't think the networking would be a huge deal. Hmm might have to test that.

 

[J-Rod]

The PBS live restore functionality is nice to have, so it might be worth attempting to increase your backup performance to improve the dirty bitmap situation.

But if that is not possible, or if you want multiple backups, then consider an agent-based backup solution. You could still use PBS to back up the VM's OS virtual disk, but exclude the data virtual disk. For Windows VMs, you have the built-in Windows backup, and for Linux have a look at BorgBackup.

Or find a Synology NAS and use Active Backup for Business, which has Windows and Linux agents.