VMs in multiple backup jobs

S

stevehughes

Active Member
Proxmox Subscriber
Apr 24, 2020
32
6
28
Brisbane, Australia
www.scalar.com.au
HI, I'm trying to work out the best way to achieve something like the following:

A backup job that performs a nightly backup of every VM on the cluster with no exclusions. This is a failsafe job to ensure that we don't forget to backup a new VM, and for many VMs a simple nightly backup is adequate.

A second job that runs hourly and keeps good historical retention, for those VMs that require it (e.g. file and database servers).

I am finding that VMs that appear in both jobs keep getting their bitmaps trashed. Re-reading the entire VM is not a dealbreaker with fast storage but its sure not optimal.

Thanks,
Steve
 
Hi,
do you maybe use the stop mode for one of you backup jobs? As stopping the VM will invalidate the dirty bitmap of the VM.
 
Please share you backup job configuration cat /etc/pve/jobs.cfg and VM configuration qm config <VMID> --current as well as the backup job task log for the backup run with invalidated dirty bitmap. Do you stop the VM in-between the runs for some reason? Are there other tasks acting on the VM in question?
 
I assume (based on the mention of different retention settings) that you are using different targets for those jobs? PVE currently only keeps one bitmap per disk for PBS backups, so if you

- switch target
- delete the last snapshot on the previous target
- change encryption mode or key

that bitmap has to be invalidated. other than overhead (both memory for keeping the bitmap around, and the tracking itself as part of the I/O path) I don't think there are any reasons to not keep multiple bitmaps (one per target) around - but it would probably require some sort of interface to also clean individual backup targets up without forcing users to clear them all ;)
 
  • Like
Reactions: Chris
Yes Fabian. Same PBS datastore, but separate namespaces. I did this to avoid having the long-term retention wiped out by the shorter retention settings of the default job.

Keeping multiple bitmaps sounds ideal. I suppose a workaround would be exclude from the default job any VMs that are included in the second job, but it defeats the security of knowing that even if someone stuffs up there will always be at least a default backup regime for every VM.
 
You must log in or register to reply here.
Top Bottom