Understanding backup on zfs

[Valerio Pachera]

Hi all, I have a guest (id 101) with a single vdi hosted on a zfs storage named 'meccanico'.
I created a backup job with 'Stop' option and gzip compression.



I noticed that the guest has been shutdown and restarted right after.
Backup is actually running but I was expecting the guest to stay off, till the end of the backup.
Otherwise, there should be at least an active snapshot.
So I tried to list zfs snapshots:

zpool get listsnapshots
NAME   PROPERTY       VALUE      SOURCE
rpool  listsnapshots  off        default
tank   listsnapshots  off        default

zfs list -t snapshot
no datasets available

zfs list
NAME                       USED  AVAIL  REFER  MOUNTPOINT
rpool                      144G  80.8G    96K  /rpool
rpool/ROOT                5.12G  80.8G    96K  /rpool/ROOT
rpool/ROOT/pve-1          5.12G  80.8G  5.12G  /
rpool/data                 130G  80.8G    96K  /rpool/data
rpool/data/vm-100-disk-1   130G  80.8G   130G  -
rpool/swap                8.50G  86.6G  2.67G  -
tank                       708G   191G    96K  /tank
tank/vm-100-disk-1         619G   649G   161G  -
tank/vm-100-disk-2        24.8G   214G  2.47G  -
tank/vm-101-disk-1        64.0G   213G  42.6G  -

Consider I'm not familiar yet with zfs, so ask me more detail I have to report.





Proxmox 5.0-23/af4267bf.

Thank you.

 

[fabian]

backups of VMs (as opposed to containers) use a special mechanism to track which parts of the disks change. you only need to start this mechanism at some point in time when there is no disk-activity / you have a consistent state of the disks - either because the VM is not running, or suspended, or by triggering a snapshot. so the stop mode does the following:

1. shutdown VM
2. start VM in a paused state (no disk activity!)
3. start backup mechanism
4. resume VM

now both the guest OS and the backup job are running and accessing the disks, but when the guest OS writes something, it is transparently buffered until the old overwritten part has been backed up. since our backup format supports writing out-of-order, these parts can be prioritized to be backed up "next".

this allows us to have backups with no (snapshot) to very little (suspend) to little (stop) downtime, and no need for temporary copies of disks or their data besides a bit of buffering of the changed data in memory. if you want more technical details, check out https://git.proxmox.com/?p=pve-qemu.git;a=blob;f=backup.txt and https://git.proxmox.com/?p=pve-qemu.git;a=blob;f=vma_spec.txt

 

[Valerio Pachera]

Cool!
Thank you for your reply.