Backup CT Change Detection Mode Override

[Elliott Partridge]

I am very happy to have finally discovered the "new" (to me) CT change detection modes for backup (see https://forum.proxmox.com/threads/pbs-client-change-detection-mode.150538/). This has vastly decreased my ~4TB file server backup from 7 hours to a few minutes.

I have a question that may be a feature request. Is there a way to configure a pool-based backup job, but have an override for this change detection mode setting for an individual container(s)? I have a "production" pool of containers with a backup job, but would like to override only the fileserver container with the Metadata change detection mode, since most of the files are static and could benefit from this mode. I could certainly switch over to the "exclude" selection mode for the job, but then any containers added to the pool would have to also be added to this job manually (I think). Or at least containers added to other pools would have to be excluded from this job.

Another question I had was whether I could alternate between doing a "Data" mode backup (probably weekly) with "Metadata" mode (nightly) and preserve the speedups I'm observing for Metadata mode? I'm thinking both use the same pxar version 2 of the archive, so I'm guessing it would reuse backup archive information seamlessly. There's some discussion in the thread above about avoiding alternating between Metadata and Default (pxar version 1) modes, since you'd lose incrementalism between the two.

 

[Chris]

I am very happy to have finally discovered the "new" (to me) CT change detection modes for backup (see https://forum.proxmox.com/threads/pbs-client-change-detection-mode.150538/). This has vastly decreased my ~4TB file server backup from 7 hours to a few minutes.

I have a question that may be a feature request. Is there a way to configure a pool-based backup job, but have an override for this change detection mode setting for an individual container(s)? I have a "production" pool of containers with a backup job, but would like to override only the fileserver container with the Metadata change detection mode, since most of the files are static and could benefit from this mode. I could certainly switch over to the "exclude" selection mode for the job, but then any containers added to the pool would have to also be added to this job manually (I think). Or at least containers added to other pools would have to be excluded from this job.

The change detection mode is set as property of the backup job, so you could create 2 pool based backup jobs, one with and another without the change detection mode set and include/exclude selected VMs accordingly. Another option would be to have 2 separate pools and backup jobs for each pool individually. Tag based filtering in jobs might make things easier here, but is not implemented yet. See https://bugzilla.proxmox.com/show_bug.cgi?id=4186

Another question I had was whether I could alternate between doing a "Data" mode backup (probably weekly) with "Metadata" mode (nightly) and preserve the speedups I'm observing for Metadata mode? I'm thinking both use the same pxar version 2 of the archive, so I'm guessing it would reuse backup archive information seamlessly. There's some discussion in the thread above about avoiding alternating between Metadata and Default (pxar version 1) modes, since you'd lose incrementalism between the two.

Yes, change-detection-mode data and metadata are using the same underlying pxar archive format version 2, which stores file metadata separately from file payload data. The modes only differ in that the metadata modes tries to skip unchanged files as compared to a metadata archive from a previous backup snapshot. So the information can be taken from a snapshot created viadata. As a matter of fact, the initial backup to a snapshot or when there is no previous snapshot for metadata mode, this acts like using the data mode. Note however that data will always read the file contents, so take longer archives with big payload size.