PVE update: Backups failed on LXCs with bind mount points

[Ansy]

All three fresh updated PVEs failed to backup LXCs with bind mount points, tasks' logs got errors like:

INFO: filesystem type on dumpdir is 'zfs' -using /var/tmp/vzdumptmp7820_210 for temporary files
INFO: Starting Backup of VM 210 (lxc)
INFO: Backup started at 2020-11-20 04:34:03
INFO: status = running
INFO: CT Name: Media
INFO: including mount point rootfs ('/') in backup
INFO: excluding bind mount point mp0 ('/srv/zfs') from backup (not a volume)
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: create storage snapshot 'vzdump'
found lock 'snapshot' trying to remove 'backup' lock
ERROR: Backup of VM 210 failed - unable to parse volume ID '/rpool/storage'
INFO: Failed at 2020-11-20 04:34:03
INFO: Backup job finished with errors

/etc/pve/lxc/210.conf:

#New Samba FileServer
#Files in /badhdd/Samba
arch: amd64
cores: 2
hostname: Samba
lock: snapshot
memory: 2048
mp0: /rpool/Samba,mp=/home
net0: name=eth0,bridge=vmbr0,firewall=1,gw=192.168.3.1,hwaddr=5E:70:0A:AA:8D:BA,ip=192.168.3.250/24,type=veth
onboot: 1
ostype: debian
rootfs: local-zfs:subvol-101-disk-0,size=8G
swap: 2048

[vzdump]
#vzdump backup snapshot
arch: amd64
cores: 2
hostname: Samba
memory: 2048
mp0: /rpool/Samba,mp=/home
net0: name=eth0,bridge=vmbr0,firewall=1,gw=192.168.3.1,hwaddr=5E:70:0A:AA:8D:BA,ip=192.168.3.250/24,type=veth
onboot: 1
ostype: debian
rootfs: local-zfs:subvol-101-disk-0,size=8G
snapstate: prepare
snaptime: 1605812402
swap: 2048

All that CTs are in locked (snapshot) state, I think updated vzdump tried to snapshot and backup bind mount points due to some bug in PVEs code.
Not updated PVEs have no [vzdump] section at all in their /etc/pve/lxc/*.conf, so I can't compare.

Spoiler: Another PVEs /etc/pve/lxc/*.conf

arch: amd64
cores: 3
hostname: Media
memory: 2048
mp0: /rpool/storage,mp=/srv/zfs
net0: name=eth0,bridge=vmbr0,firewall=1,gw=10.150.10.250,gw6=2a03:e2c0:35b:5555::1,hwaddr=F6:AE:5A:C3:74:AB,ip=10.150.10.210/8,ip6=2a03:e2c0:35b:5555::210/64,type=veth
onboot: 1
ostype: debian
rootfs: local-zfs:subvol-210-disk-0,size=8G
swap: 2048
lxc.mount.entry: /dev/fuse dev/fuse none bind,create=file

[vzdump]
#vzdump backup snapshot
arch: amd64
cores: 3
hostname: Media
memory: 2048
mp0: /rpool/storage,mp=/srv/zfs
net0: name=eth0,bridge=vmbr0,firewall=1,gw=10.150.10.250,gw6=2a03:e2c0:35b:5555::1,hwaddr=F6:AE:5A:C3:74:AB,ip=10.150.10.210/8,ip6=2a03:e2c0:35b:5555::210/64,type=veth
onboot: 1
ostype: debian
rootfs: local-zfs:subvol-210-disk-0,size=8G
snapstate: prepare
snaptime: 1605828843
swap: 2048
lxc.mount.entry: /dev/fuse dev/fuse none bind,create=file

Spoiler: Versions

proxmox-ve: 6.2-2 (running kernel: 5.4.55-1-pve) pve-manager: 6.2-15 (running version: 6.2-15/48bd51b6) pve-kernel-5.4: 6.3-1 pve-kernel-helper: 6.3-1 pve-kernel-5.3: 6.1-6 pve-kernel-5.4.73-1-pve: 5.4.73-1 pve-kernel-5.4.65-1-pve: 5.4.65-1 pve-kernel-5.4.55-1-pve: 5.4.55-1 pve-kernel-5.3.18-3-pve: 5.3.18-3 ceph-fuse: 12.2.11+dfsg1-2.1+b1 corosync: 3.0.4-pve1 criu: 3.11-3 glusterfs-client: 5.5-3 ifupdown: 0.8.35+pve1 libjs-extjs: 6.0.1-10 libknet1: 1.16-pve1 libproxmox-acme-perl: 1.0.5 libpve-access-control: 6.1-3 libpve-apiclient-perl: 3.0-3 libpve-common-perl: 6.2-4 libpve-guest-common-perl: 3.1-3 libpve-http-server-perl: 3.0-6 libpve-storage-perl: 6.2-10 libqb0: 1.0.5-1 libspice-server1: 0.14.2-4~pve6+1 lvm2: 2.03.02-pve4 lxc-pve: 4.0.3-1 lxcfs: 4.0.3-pve3 novnc-pve: 1.1.0-1 proxmox-backup-client: 1.0.1-1 proxmox-mini-journalreader: 1.1-1 proxmox-widget-toolkit: 2.3-10 pve-cluster: 6.2-1 pve-container: 3.2-3 pve-docs: 6.2-6 pve-edk2-firmware: 2.20200531-1 pve-firewall: 4.1-3 pve-firmware: 3.1-3 pve-ha-manager: 3.1-1 pve-i18n: 2.2-2 pve-qemu-kvm: 5.1.0-6 pve-xtermjs: 4.7.0-2 qemu-server: 6.2-20 smartmontools: 7.1-pve2 spiceterm: 3.1-1 vncterm: 1.6-2 zfsutils-linux: 0.8.5-pve1

 

[SnowFox]

I can confirm the same happening at my environment too.

 

[synci]

Here too.

 

[juliokele]

Temporary solution:
https://forum.proxmox.com/threads/b...r-unable-to-parse-volume-id.79334/post-351351

 

[uncle.cripple]

I can confirm the same issue. The linked temporary solution apt install pve-container=3.2-2 fixed the problem for now.

 

[Dominic]

Please keep track of https://bugzilla.proxmox.com/show_bug.cgi?id=3161

 

[Dominic]

There is now a patch
https://lists.proxmox.com/pipermail/pve-devel/2020-November/046044.html

I suppose it is applied soon and then it will not be long until the packaged version arrives in pve-test. It would be great if you could test it then!