Problem backing up container

S

struppu

New Member
Dec 4, 2025
10
0
1
Yesterday I updated Proxmox from 8 to 9. Last night I encauntered errors backing up one of my Containers to a Proxmox Backup Server

Error Message:
Code: 
NFO: starting new backup job: vzdump 101 --all 0 --prune-backups 'keep-daily=7,keep-monthly=12,keep-weekly=4,keep-yearly=999' --mode snapshot --storage duckpbs --notes-template '{{guestname}}' --node duckmox --fleecing 0 --notification-mode notification-system
INFO: Starting Backup of VM 101 (lxc)
INFO: Backup started at 2025-12-29 12:59:10
INFO: status = running
INFO: CT Name: duckcloud
INFO: including mount point rootfs ('/') in backup
INFO: including mount point mp0 ('/srv') in backup
INFO: found old vzdump snapshot (force removal)
  Logical volume "snap_vm-101-disk-0_vzdump" successfully removed.
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: suspend vm to make snapshot
INFO: create storage snapshot 'vzdump'
  Logical volume "snap_vm-101-disk-0_vzdump" created.
zfs_mount_at() failed: directory is not emptyumount: /mnt/vzsnap0/srv: not mounted.
command 'umount -l -d /mnt/vzsnap0/srv' failed: exit code 32
INFO: resume vm
INFO: guest is online again after <1 seconds
ERROR: Backup of VM 101 failed - command 'mount -o ro -t zfs zfs_storage/subvol-101-disk-0@vzdump /mnt/vzsnap0//srv' failed: exit code 2
INFO: Failed at 2025-12-29 12:59:10
INFO: Backup job finished with errors
INFO: notified via target `mail-to-root`
TASK ERROR: job errors

Backups of the other containers work as expected it's just this one container that does not work, even though two more containers also have a mountpoint on the zfs storage

The container itself works just normal, only backing up is not possible

Anyone got an idea what the problem might be, because for me the error message does not make sense
Thanks in advance
 
This error is caused by a leftover mount directory from a failed snapshot, which started happening after the Proxmox 9 upgrade. The key message is:


zfs_mount_at() failed: directory is not empty
Click to expand...

This means Proxmox is trying to mount the snapshot, but the mount directory already contains files from a previous failed backup.

To Fix this error​

  1. Stop the container

pct stop 101

  1. Check and clean leftover snapshot mounts
ls -lah /mnt/vzsnap0


If you see directories like srv or old data inside:

umount -lf /mnt/vzsnap0/srv 2>/dev/null
rm -rf /mnt/vzsnap0/srv


  1. Remove any leftover ZFS snapshots
zfs list -t snapshot | grep subvol-101
zfs destroy zfs_storage/subvol-101-disk-0@vzdump


  1. Start the container again


pct start 101


  1. Run the backup manually to test
vzdump 101 --mode snapshot --storage duckpbs
 
Last edited:
  • Like
Reactions: leesteken
thank you for your reply

I tried your suggestions:
after stopping the container
ls -lah /mnt/vzsnap0
showed me
root@duckmox:/# ls -lah /mnt/vzsnap0/
total 8.0K
drwxr-xr-x 2 root root 4.0K Dec 27 21:54 .
drwxr-xr-x 3 root root 4.0K Dec 27 21:54 .

so i could neither delete or unmount anything in that folder.

zfs list -t snapshot | grep subvol-101
outputs
zfs_storage/subvol-101-disk-0@vzdump 384K - 10.3G -

after destroying the snapshot with:
zfs destroy zfs_storage/subvol-101-disk-0@vzdump
the output of
zfs list -t snapshot | grep subvol-101
is
no datasets available

after starting the container manually running the backup gives the following output.
I run it twice because in after the first run I got an zfs error
zfs error: could not find any snapshots to destroy; check snapshot names.

Code: 
vzdump 101 --mode snapshot --storage duckpbs

INFO: starting new backup job: vzdump 101 --storage duckpbs --mode snapshot
INFO: Starting Backup of VM 101 (lxc)
INFO: Backup started at 2025-12-29 16:40:03
INFO: status = running
INFO: CT Name: duckcloud
INFO: including mount point rootfs ('/') in backup
INFO: including mount point mp0 ('/srv') in backup
INFO: found old vzdump snapshot (force removal)
  Logical volume "snap_vm-101-disk-0_vzdump" successfully removed.
zfs error: could not find any snapshots to destroy; check snapshot names.
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: suspend vm to make snapshot
INFO: create storage snapshot 'vzdump'
  Logical volume "snap_vm-101-disk-0_vzdump" created.
zfs_mount_at() failed: directory is not emptyumount: /mnt/vzsnap0/srv: not mounted.
command 'umount -l -d /mnt/vzsnap0/srv' failed: exit code 32
INFO: resume vm
INFO: guest is online again after 7 seconds
ERROR: Backup of VM 101 failed - command 'mount -o ro -t zfs zfs_storage/subvol-101-disk-0@vzdump /mnt/vzsnap0//srv' failed: exit code 2
INFO: Failed at 2025-12-29 16:40:10
INFO: Backup job finished with errors
INFO: notified via target `mail-to-root`
job errors


vzdump 101 --mode snapshot --storage duckpbs

INFO: starting new backup job: vzdump 101 --mode snapshot --storage duckpbs
INFO: Starting Backup of VM 101 (lxc)
INFO: Backup started at 2025-12-29 16:40:51
INFO: status = running
INFO: CT Name: duckcloud
INFO: including mount point rootfs ('/') in backup
INFO: including mount point mp0 ('/srv') in backup
INFO: found old vzdump snapshot (force removal)
  Logical volume "snap_vm-101-disk-0_vzdump" successfully removed.
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: suspend vm to make snapshot
INFO: create storage snapshot 'vzdump'
  Logical volume "snap_vm-101-disk-0_vzdump" created.
zfs_mount_at() failed: directory is not emptyumount: /mnt/vzsnap0/srv: not mounted.
command 'umount -l -d /mnt/vzsnap0/srv' failed: exit code 32
INFO: resume vm
INFO: guest is online again after 1 seconds
ERROR: Backup of VM 101 failed - command 'mount -o ro -t zfs zfs_storage/subvol-101-disk-0@vzdump /mnt/vzsnap0//srv' failed: exit code 2
INFO: Failed at 2025-12-29 16:40:52
INFO: Backup job finished with errors
INFO: notified via target `mail-to-root`
job errors

so this unfortunately did not solve the problem, do you have any other ideas?
 
I have one more question:
Why does proxmox want to mount the snapshot in /mnt/vzsnap0/srv
is it because my mountpoint mp0 ist mounted as /srv in the container?

The strange thing is that two of my other three containers have a mountpoint mounted under /srv but they get backed up without any problem

I also tried to create the srv directory under /mnt/vzsnap0 but then the backup fails with another error
Code: 
INFO: starting new backup job: vzdump 101 --storage duckpbs --mode snapshot
INFO: Starting Backup of VM 101 (lxc)
INFO: Backup started at 2025-12-30 00:15:42
INFO: status = running
INFO: CT Name: duckcloud
INFO: including mount point rootfs ('/') in backup
INFO: including mount point mp0 ('/srv') in backup
INFO: found old vzdump snapshot (force removal)
  Logical volume "snap_vm-101-disk-0_vzdump" successfully removed.
ERROR: Backup of VM 101 failed - mount point '/mnt/vzsnap0' not empty
INFO: Failed at 2025-12-30 00:15:42
INFO: Backup job finished with errors
INFO: notified via target `mail-to-root`
job errors

mount point '/mnt/vzsnap0' not empty

Now I tried to clone the container and run a backup on the clone and it worked just fine

now I am really confused for my understanding the clone should have the same problem as the original or am I wrong?

I had one more idea, since I tried the backup on the clone in stopped state, I tried it also with the container running with the result that the backup failed with the same error as on the original.

The problem seems that this container can only be backuped in stoped state - but this is only since I updatet proxmox from 8.4 to 9.1
 
Last edited:
Hi,
can you check ls -al /mnt/vzsnap0/ to see if the directory is empty before the backup? If it's not, it might be the issue. If it is, please share the container configuration pct config 101.
 
The directory is empty - see output below
[B]ls -al /mnt/vzsnap0/[/B]
total 8
drwxr-xr-x 2 root root 4096 Dec 30 00:16 .
drwxr-xr-x 3 root root 4096 Dec 29 23:53 ..


config for container 101
[B]pct config 101[/B]
arch: amd64
cores: 2
features: nesting=1
hostname: duckcloud
memory: 2048
mp0: zfs_storage:subvol-101-disk-0,mp=/srv,backup=1,size=128G
net0: name=eth0,bridge=vmbr0,firewall=1,hwaddr=BC:24:11:11:91:BD,ip=dhcp,type=veth
onboot: 1
ostype: debian
rootfs: local-lvm:vm-101-disk-0,size=8G
swap: 512
unprivileged: 1

for comparison container 102
[B]pct config 102[/B]
arch: amd64
cores: 1
features: nesting=1
hostname: duckgit
memory: 512
mp0: zfs_storage:subvol-102-disk-0,mp=/srv,backup=1,size=4G
net0: name=eth0,bridge=vmbr0,firewall=1,hwaddr=BC:24:11:87:86:BE,ip=dhcp,type=veth
onboot: 1
ostype: debian
rootfs: local-lvm:vm-102-disk-0,size=8G
swap: 512
unprivileged: 1

both have a mountpoint (mp0) a zfs_subvolume (on the same physical ssd) which ist mounted to /srv
both container use the mountpoint to store a mysql datadirectory
101 also hosts a nextcloud datadirectory whreas 102 git repositories

as mentioned before until my upgrade to proxmox ve 9.1 backups in snapshot mode worked for all containers
since the snaptshot update backups for container 101 stopped working printing the above mentioned error, backing up in stopped mode works
 
You must log in or register to reply here.
Top Bottom