Backup fails when (additional) container disk is on ZFS ZVOL

[yummiweb]

Proxmox VE 9.1.4
Proxmox Backup Server 4.10

(The problem did not occur under PVE 8 / PBS 3)

Object: Container
Disk 0: "root"
Disk 1: "mp0" (/mnt/db_backups)
Disk locations: ZFS ZVOL

Problem:
Backup to PBS fails with the following error:

ERROR: Backup of VM 164001 failed - command 'mount -o ro -t zfs ZFS_POOL/zvol/subvol-164001-disk-1@vzdump /mnt/vzsnap0//mnt/db_backups' failed: exit code 2

(Note the double "/" in the path after vzsnap0)

Details:

vzdump 164001 --mode snapshot --storage PBS_LOCAL --remove 0 --notes-template '{{guestname}}' --node base16 --notification-mode notification-system
INFO: Starting Backup of VM 164001 (lxc)
INFO: status = running
INFO: CT Name: admv1
INFO: including mount point rootfs ('/') in backup
INFO: including mount point mp0 ('/mnt/db_backups') in backup
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: suspend vm to make snapshot
INFO: create storage snapshot 'vzdump'
INFO: resume vm
INFO: guest is online again after <1 seconds
ERROR: Backup of VM 164001 failed - command 'mount -o ro -t zfs ZFS_POOL/zvol/subvol-164001-disk-1@vzdump /mnt/vzsnap0//mnt/db_backups' Failed: Exit code 2

In the end, an undeleted snapshot "vzsnap" of the container remains.

Current workaround:

I moved "mp0" (/mnt/db_backups) from ZVOL to the file-based area of ZFS_POOL – the backup works from there for now.

regards Yummiweb

 

[Impact]

Please share pct config 164001.

from ZVOL

Can you elaborate on this? Why/how did you use a ZVOL?

 

[yummiweb]

Please share pct config 164001.

Can you elaborate on this? Why/how did you use a ZVOL?

The "workaround" unfortunately has a significant drawback: (PVE) snapshots of the container can no longer be created if one of the drives is not located on a ZVOL.

Which is also the reason for storing the container on a ZVOL – to create snapshots of the container.

What makes the situation truly problematic at the moment is this:

If you choose to use a ZVOL, you have no backups (until this issue is resolved).

If you choose to use a workaround instead of a ZVOL, you have no snapshots.

I would like to report this directly as a bug, but I can't find where to do so.

 

[yummiweb]

Please share pct config 164001.

This is the configuration at the time of the error (already under PVE9) - both drives on the ZVOL.

arch: amd64
cores: 2
cpulimit: 1
features: nesting=1
hostname: myhost
memory: 2048
mp0: POOL_A_ZVOL:subvol-164001-disk-1,mp=/mnt/db_backups,backup=1,mountoptions=discard;lazytime,size=8G
nameserver: 192.168.222.1
net0: name=eth0,bridge=vmbr1621,gw=192.168.222.1,hwaddr=DE:49:22:22:02:01,ip=192.168.222.12/32,tag=12,type=veth
ostype: debian
rootfs: POOL_A_ZVOL:subvol-164001-disk-0,mountoptions=discard;lazytime,size=16G
searchdomain: mydomain.internal
startup: order=4,down=180
swap: 4096
unprivileged: 1

 

[Impact]

I think we have different opinions on what a ZVOL is. Can you share cat /etc/pve/storage.cfg in a code block?

 

[yummiweb]

I think we have different opinions on what a ZVOL is. Can you share cat /etc/pve/storage.cfg in a code block?

This is a ZVOL, right?

zfspool: POOL_A_ZVOL
pool ZFS_POOL_A/zvol
blocksize 16k
content images,rootdir
mountpoint /ZFS_POOL_A/zvol
sparse 1

And that is a "normal" Directory Storage on ZFS:

dir: POOL_A_FILES
path /mnt/MNTNAME/zfs_poo_a
content rootdir,images
prune-backups keep-all=1
shared 0

What are yor diffrent opinion?

Remeber my first Post, please:
The problem persist since PVE9, on PVE8 there was no Problem with Backups

 

[Impact]

I'd consider that a dataset which makes this pretty confusing. Please share zfs list -ospace,type. Only volume is a ZVOL.
Can you share findmnt -T /mnt/MNTNAME/zfs_poo_a and findmnt -T /mnt/db_backups too? I need to get a grasp of your setup before I can consider what might be wrong.
Please use code blocks when sharing formatted text so this formatting is preserved.