I have a lab setup at home. Have a little n100 PC doing PBS backups of a PVE server and everything has been going great and just starting recently getting this error. Ive reviewed things and chatgpt said do a few things to check for free space and space isn't an issue but the backup continues to fail. It's only failing on the Komo1_VPN backup, all other LXC and VMs running on the host backup as intended as you can see from the logs for the Scrypted VM.
appreciate any help with this one, id like to make sure this LXC starts backup up again as there a few services on that machine I use daily.
Thanks!
appreciate any help with this one, id like to make sure this LXC starts backup up again as there a few services on that machine I use daily.
Thanks!
Bash:
Loga sent via email from proxmox for failing backup
104: 2025-06-05 19:37:50 INFO: Starting Backup of VM 104 (lxc)
104: 2025-06-05 19:37:50 INFO: status = running
104: 2025-06-05 19:37:50 INFO: CT Name: komo1_vpn
104: 2025-06-05 19:37:50 INFO: including mount point rootfs ('/') in backup
104: 2025-06-05 19:37:50 INFO: mode failure - some volumes do not support snapshots
104: 2025-06-05 19:37:50 INFO: trying 'suspend' mode instead
104: 2025-06-05 19:37:50 INFO: backup mode: suspend
104: 2025-06-05 19:37:50 INFO: ionice priority: 7
104: 2025-06-05 19:37:50 INFO: CT Name: komo1_vpn
104: 2025-06-05 19:37:50 INFO: including mount point rootfs ('/') in backup
104: 2025-06-05 19:37:50 INFO: starting first sync /proc/4421/root/ to /var/tmp/vzdumptmp1254577_104
104: 2025-06-05 19:51:23 INFO: first sync finished - transferred 99.57G bytes in 813s
104: 2025-06-05 19:51:23 INFO: suspending guest
104: 2025-06-05 19:51:23 INFO: starting final sync /proc/4421/root/ to /var/tmp/vzdumptmp1254577_104
104: 2025-06-05 19:51:34 ERROR: rsync: [receiver] write failed on "/var/tmp/vzdumptmp1254577_104/opt/stacks/nzbhydra2/data/database/nzbhydra.mv.db": No space left on device (28)
104: 2025-06-05 19:51:34 ERROR: rsync error: error in file IO (code 11) at receiver.c(381) [receiver=3.2.7]
104: 2025-06-05 19:51:34 INFO: resume vm
104: 2025-06-05 19:51:34 INFO: guest is online again after 11 seconds
104: 2025-06-05 19:52:15 ERROR: Backup of VM 104 failed - command 'rsync --stats -h -X -A --numeric-ids -aH --delete --no-whole-file --sparse --one-file-system --relative '--exclude=/tmp/?*' '--exclude=/var/tmp/?*' '--exclude=/var/run/?*.pid' /proc/4421/root//./ /var/tmp/vzdumptmp1254577_104' failed: exit code 11
Bash:
No errors on the Scrypted VM
10443: 2025-06-05 19:54:56 INFO: Starting Backup of VM 10443 (lxc)
10443: 2025-06-05 19:54:56 INFO: status = running
10443: 2025-06-05 19:54:56 INFO: CT Name: scrypted
10443: 2025-06-05 19:54:56 INFO: including mount point rootfs ('/') in backup
10443: 2025-06-05 19:54:56 INFO: including mount point mp0 ('/root/.scrypted/volume') in backup
10443: 2025-06-05 19:54:56 INFO: excluding volume mount point mp1 ('/mnt/pve/ssd2tb') from backup (disabled)
10443: 2025-06-05 19:54:56 INFO: backup mode: snapshot
10443: 2025-06-05 19:54:56 INFO: ionice priority: 7
10443: 2025-06-05 19:54:56 INFO: suspend vm to make snapshot
10443: 2025-06-05 19:55:58 INFO: create storage snapshot 'vzdump'
10443: 2025-06-05 19:56:52 INFO: resume vm
10443: 2025-06-05 19:56:52 INFO: guest is online again after 116 seconds
10443: 2025-06-05 19:56:52 INFO: creating Proxmox Backup Server archive 'ct/10443/2025-06-05T23:54:56Z'
10443: 2025-06-05 19:56:52 INFO: set max number of entries in memory for file-based backups to 1048576
10443: 2025-06-05 19:56:52 INFO: run: lxc-usernsexec -m u:0:100000:65536 -m g:0:100000:65536 -- /usr/bin/proxmox-backup-client backup --crypt-mode=none pct.conf:/var/tmp/vzdumptmp1254577_10443/etc/vzdump/pct.conf root.pxar:/mnt/vzsnap0 --include-dev /mnt/vzsnap0/./ --include-dev /mnt/vzsnap0/./root/.scrypted/volume --skip-lost-and-found --exclude=/tmp/?* --exclude=/var/tmp/?* --exclude=/var/run/?*.pid --backup-type ct --backup-id 10443 --backup-time 1749167696 --entries-max 1048576 --repository root@pam@192.168.1.26:ssd-zfs --ns pve
10443: 2025-06-05 19:56:53 INFO: Starting backup: [pve]:ct/10443/2025-06-05T23:54:56Z
10443: 2025-06-05 19:56:53 INFO: Client name: pve
10443: 2025-06-05 19:56:53 INFO: Starting backup protocol: Thu Jun 5 19:56:53 2025
10443: 2025-06-05 19:56:53 INFO: Downloading previous manifest (Thu Jun 5 11:56:30 2025)
10443: 2025-06-05 19:56:53 INFO: Upload config file '/var/tmp/vzdumptmp1254577_10443/etc/vzdump/pct.conf' to 'root@pam@192.168.1.26:8007:ssd-zfs' as pct.conf.blob
10443: 2025-06-05 19:56:53 INFO: Upload directory '/mnt/vzsnap0' to 'root@pam@192.168.1.26:8007:ssd-zfs' as root.pxar.didx
10443: 2025-06-05 19:57:53 INFO: processed 3.526 GiB in 1m, uploaded 11.301 MiB
10443: 2025-06-05 19:58:53 INFO: processed 6.923 GiB in 2m, uploaded 20.644 MiB
10443: 2025-06-05 19:59:53 INFO: processed 11.345 GiB in 3m, uploaded 20.644 MiB
10443: 2025-06-05 20:00:52 INFO: root.pxar: had to backup 66.334 MiB of 14.596 GiB (compressed 10.556 MiB) in 239.46 s (average 283.667 KiB/s)
10443: 2025-06-05 20:00:52 INFO: root.pxar: backup was done incrementally, reused 14.532 GiB (99.6%)
10443: 2025-06-05 20:00:52 INFO: Uploaded backup catalog (5.549 MiB)
10443: 2025-06-05 20:00:52 INFO: Duration: 239.52s
10443: 2025-06-05 20:00:52 INFO: End Time: Thu Jun 5 20:00:52 2025
10443: 2025-06-05 20:00:52 INFO: adding notes to backup
10443: 2025-06-05 20:00:52 INFO: prune older backups with retention: keep-daily=3, keep-hourly=6, keep-last=10, keep-monthly=1, keep-weekly=1, keep-yearly=1
10443: 2025-06-05 20:00:53 INFO: running 'proxmox-backup-client prune' for 'ct/10443'
10443: 2025-06-05 20:00:53 INFO: pruned 0 backup(s)
10443: 2025-06-05 20:00:58 INFO: cleanup temporary 'vzdump' snapshot
10443: 2025-06-05 20:00:59 INFO: Finished Backup of VM 10443 (00:06:03)
PVE HOST
root@pve:/var# df -h
Filesystem Size Used Avail Use% Mounted on
udev 63G 0 63G 0% /dev
tmpfs 13G 3.3M 13G 1% /run
/dev/mapper/pve-root 78G 47G 28G 63% /
tmpfs 63G 34M 63G 1% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
/dev/sdd1 1.8T 40K 1.7T 1% /mnt/pve/ssd2tb
/dev/fuse 128M 36K 128M 1% /etc/pve
192.168.1.12:/volume1/plex 23T 14T 9.5T 59% /mnt/pve/plex
192.168.1.12:/volume1/ISO_Library 23T 14T 9.5T 59% /mnt/pve/iso_library
192.168.1.12:/volume1/nas-nvr-storage 23T 14T 9.5T 59% /mnt/pve/nas-nvr-storage
tmpfs 13G 0 13G 0% /run/user/0
LXC that won't backup
root@komo1vpn:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/loop0 69G 30G 36G 46% /
none 492K 4.0K 488K 1% /dev
tmpfs 63G 0 63G 0% /dev/shm
tmpfs 26G 2.8M 26G 1% /run
tmpfs 5.0M 0 5.0M 0% /run/lock
192.168.1.12:/volume1/backups 23T 14T 9.5T 59% /mnt/backups
192.168.1.12:/volume1/plex 23T 14T 9.5T 59% /mnt/plex
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/9442a72a72e7f2018aff70861c0fa362e28c561688351c366ade073ffad80f1e/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/96eb1bab88b24a94bc38f8918ef9329ca241a660702fae9eb02f0cb7cc6234d3/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/3f61d23b66cd33796ecd3f32d02e0290f3c90d73f6df1b4a03cdbe25987cf309/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/12713b4438b8f347e3ad7e05c8287d922d98caf582a68e9b65deb4ee728d57e7/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/5e834a55f5af302373e406d6f49ee80cd34db451546b8fe0d8decc432c65f17c/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/a7c12e306e0a12a1310556795a58985350c171b0f26c71dab8c9ed778b85dbdc/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/a9f3365882349cfa58efd7c9c9ba7485629e5adba59886be14642eef59e99d02/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/cfd0cd7ed1fd4a87a3562d66c4a6d10781b894b6159fa021d59489e70739efed/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/01c5a6bcdde59ecf27e28b049d9984d5064bea44f966bcf39de4676a352cc447/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/7bcd5bb5e99bb667ee4048f0eb0466136893c4c65658df020184b76f60f5ec46/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/8678e2f8d7dab5129377767250aa6ea16813391eb3e40dc12ad99d8a7f16ac56/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/3177c28219d824eb339db780686bb7f90fda87b67ad166e3c47d2205c981fca6/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/50966df560f5b09ddcf1d17837ed780c548f9c5bef6795d83c183351347a1a73/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/46117d8d4f1436ca0b8c60a72463c6cda7689ad1b0eb4f57bf2f3dbcd10ad812/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/80e88b3a5f8ccb475194db63e372c3bbee41d212d321bf96085aae6615f842f4/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/e861dabbf539b93e05304b6f76bc2042325899ff67230c00d12c09ec871c8aa1/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/f6e54ac51fefd693bccc6f705da77a7087eaf939253a39ad682df2e37d7eae8f/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/2a1fae0889a9a6c20834bfda23c3e4d845e21bb33db23551f86cd93329bffceb/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/357debe5ae6713a835a532d94a74d85cd527a89eca237aa86145b15f128f7fae/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/608b67bc6c78a1cd8e15012bf8c84c81dfc283d0a72977366f3817f061d01b67/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/2eb9595373ad4d5c2c57ed36742e14793e2893f3bd637f990502da0b0bbddc9b/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/4648d21ef386c06a385ce51e660c17562c8a69f34d55ee754f253d9ef8afab7a/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/7175e9780463e629619dd823ba7ed735589bde128b2b9ed7d448a3fa07b45fbe/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/8395e139d7595a44526ba997f101676cca06fc18691b8466b0d5ce9677887573/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/76dc2e5efe61e6e2086edaedf221eb73f0d29f8e48e2ee7b9b86ea5496caf239/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/eaa00eb57b6af30f4844fde9d26d934cd18d886428d251333017716d4e587b01/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/1cb692172b4db2456d305e4f6290b754472651f248ea44a4332c3e7d733cf197/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/a25eaf5d619a6bc879d63566e9800eafcfa879f8a05fc3356f44bcf1b7151aad/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/64112eccfa324d10cb0fe42852670142025f0834613371b2c3251712aab972ea/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/f7400d429ded83e43297c5962adecd92c99ccb350c16916eae10eedfa2d640cb/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/698e8e1af9de10ac75b474f80f11cd015b979dcbaa5a758695bdebb5549aa019/merged
overlay 69G 30G 36G 46% /var/lib/docker/overlay2/cbd5add8a716fa3149dd156baf5128be5f4c128a41dfa0d9290323bea282d7ea/merged
tmpfs 13G 0 13G 0% /run/user/0