PBS Incremental Backup

[huitheme]

In my PVE environment, I have either a Debian virtual machine or a Debian LXC container—both with roughly the same storage capacity, say, 20GB.

I would like to perform backups every six hours, and I have a dedicated PBS server available.

My question is: Should I set up a Debian virtual machine, or opt for a Debian LXC container? Which option generates the least amount of disk read/write activity during the backup process?

My core requirement is this: The backup process absolutely cannot involve a full read and write of the entire dataset every single time, as that would drastically increase the overall disk I/O load.

 

[leesteken]

If you only have one (repeated) backup job for a VM then Proxmox keeps track of which disk blocks have changed and PBS does not need to read the other disk blocks. This is reset when you stop and restart the VM, reboot or another backup jobs runs.

 

[Johannes S]

And for lxcs (and file-based backups) you can set the change-detect-mode to metadata which will be used to detect already backed up data and should speedup the backups accordingly.

The real question you should ask yourself which kind of workloads you plan to run inside your Debian guest. For docker or podman containers I would always recommend a Debian VM except you have a really good reason not too.

LXCs however are more lightweight and thus great for self-containing services like AdGuard or Pihole. Another usecase is to use a lxc as fileserver. YMMV

 

[LnxBil]

I would assume that backing up only LX(C) containers would save space in comparison to mixed or even only VMs. Containers are backed up on a file level, therefore cohrent files are backed up in one go as one file and are not stripped across multiple blocks in a VM. I have no checked this, but I would assume this due to the file vs. block based backup setting. The deduplication rate would also be pretty huge if the containers share the OS. I tested this with ZFS and deduplication and the space savings were huge.

EDIT: typos

 

[spirit]

In my PVE environment, I have either a Debian virtual machine or a Debian LXC container—both with roughly the same storage capacity, say, 20GB.

I would like to perform backups every six hours, and I have a dedicated PBS server available.

My question is: Should I set up a Debian virtual machine, or opt for a Debian LXC container? Which option generates the least amount of disk read/write activity during the backup process?

My core requirement is this: The backup process absolutely cannot involve a full read and write of the entire dataset every single time, as that would drastically increase the overall disk I/O load.

lxc backup is scanning all files to see if metadatas (modification time) have changed , so if you want millions of small files, it can put load on the disks even if you don't have changes.

vm backup use internal qemu dirty-bitmap tracking, so when backup is starting, it's directly known which vm block it need to save. (and it's also a lot faster to save some block vs millions of small files).

 

[Johannes S]

lxc backup is scanning all files to see if metadatas (modification time) have changed , so if you want millions of small files, it can put load on the disks even if you don't have changes.

Shouldn't this be mitigated by using the metadata mode for change detection of LXCs? Or is my understanding of following part of the docs wrong?

CT Change Detection Mode
Setting the change detection mode defines the encoding format for the pxar archives and how changed, and unchanged files are handled for container backups with Proxmox Backup Server as the target.

The change detection mode option can be configured for individual backup jobs in the Advanced tab while editing a job. Further, this option can be set as node-wide fallback via the configuration options.

There are 3 change detection modes available:

Mode Description
Default Read and encode all files into a single archive, using the pxar format version 1.
Data: Read and encode all files, but split data and metadata into separate streams, using the pxar format version 2.
Metadata: Split streams and use archive format version 2 like Data, but use the metadata archive of the previous snapshot (if one exists) to detect unchanged files, and reuse their data chunks without reading file contents from disk, whenever possible.

To perform a backup using the change detecation mode metadata you can run vzdump 123 --storage pbs-storage --pbs-change-detection-mode metadata
Note Backups of VMs or to storage backends other than Proxmox Backup Server are not affected by this setting.
https://pve.proxmox.com/pve-docs/chapter-vzdump.html#_ct_change_detection_mode