Since we updated our Proxmox cluster from 8.0.4 to 8.4.14 we are experiencing problems with scheduled back-up jobs. For our back-ups we use Proxmox Back-up Server version 3.2-2. The follow occurs: The back-up job will crash on a random VM while taking a snapshot and backing up the data to PBS, it stop at a random percentage. Because of this the VM gets corrupted and needs to be restored from an earlier back-up. See log below
INFO: starting new backup job: vzdump 1001 1002 1017 1018 1012 --fleecing 0 --node have-hst03 --all 0 --prune-backups 'keep-last=14,keep-monthly=6' --storage pmbs-hdc --mailnotification failure --notes-template '{{guestname}}' --mode snapshot --mailto
INFO: Starting Backup of VM 1012 (qemu)
INFO: Backup started at 2025-11-12 11:20:11
INFO: status = running
INFO: VM Name: HDC-LNMS01-DEB12
INFO: include disk 'scsi0' 'Clusterdata:1012/vm-1012-disk-1.qcow2' 32G
INFO: include disk 'efidisk0' 'Clusterdata:1012/vm-1012-disk-0.qcow2' 528K
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating Proxmox Backup Server archive 'vm/1012/2025-11-12T10:20:11Z'
INFO: issuing guest-agent 'fs-freeze' command
INFO: issuing guest-agent 'fs-thaw' command
INFO: started backup task 'b9219771-2526-41aa-a3cc-fd3b7eba3746'
INFO: resuming VM again
INFO: efidisk0: dirty-bitmap status: OK (drive clean)
INFO: scsi0: dirty-bitmap status: OK (672.0 MiB of 32.0 GiB dirty)
INFO: using fast incremental mode (dirty-bitmap), 672.0 MiB dirty of 32.0 GiB total
INFO: 100% (672.0 MiB of 672.0 MiB) in 3s, read: 224.0 MiB/s, write: 218.7 MiB/s
INFO: backup was done incrementally, reused 31.36 GiB (97%)
INFO: transferred 672.00 MiB in 7 seconds (96.0 MiB/s)
INFO: adding notes to backup
INFO: prune older backups with retention: keep-last=14, keep-monthly=6
INFO: running 'proxmox-backup-client prune' for 'vm/1012'
INFO: pruned 1 backup(s) not covered by keep-retention policy
INFO: Finished Backup of VM 1012 (00:00:09)
INFO: Backup finished at 2025-11-12 11:20:20
INFO: Starting Backup of VM 1017 (qemu)
INFO: Backup started at 2025-11-12 11:20:20
INFO: status = running
INFO: VM Name: HDC-WS01-DEB13
INFO: include disk 'scsi0' 'Clusterdata:1017/vm-1017-disk-1.qcow2' 128G
INFO: include disk 'efidisk0' 'Clusterdata:1017/vm-1017-disk-0.qcow2' 528K
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating Proxmox Backup Server archive 'vm/1017/2025-11-12T10:20:20Z'
INFO: issuing guest-agent 'fs-freeze' command
INFO: issuing guest-agent 'fs-thaw' command
INFO: started backup task 'fe37a194-8845-4988-9e83-11a4cc5ec0f5'
INFO: resuming VM again
INFO: efidisk0: dirty-bitmap status: OK (drive clean)
INFO: scsi0: dirty-bitmap status: existing bitmap was invalid and has been cleared
INFO: using fast incremental mode (dirty-bitmap), 128.0 GiB dirty of 128.0 GiB total
INFO: 1% (2.4 GiB of 128.0 GiB) in 3s, read: 816.0 MiB/s, write: 50.7 MiB/s
INFO: 3% (4.9 GiB of 128.0 GiB) in 6s, read: 846.7 MiB/s, write: 24.0 MiB/s
INFO: 4% (5.2 GiB of 128.0 GiB) in 10s, read: 74.0 MiB/s, write: 38.0 MiB/s
INFO: 5% (6.5 GiB of 128.0 GiB) in 52s, read: 31.6 MiB/s, write: 19.5 MiB/s
INFO: 6% (8.2 GiB of 128.0 GiB) in 1m 4s, read: 145.7 MiB/s, write: 1.3 MiB/s
INFO: 19% (24.4 GiB of 128.0 GiB) in 1m 7s, read: 5.4 GiB/s, write: 0 B/s
INFO: 32% (41.8 GiB of 128.0 GiB) in 1m 10s, read: 5.8 GiB/s, write: 6.7 MiB/s
INFO: 43% (56.0 GiB of 128.0 GiB) in 1m 13s, read: 4.7 GiB/s, write: 4.0 MiB/s
INFO: 58% (74.5 GiB of 128.0 GiB) in 1m 16s, read: 6.2 GiB/s, write: 98.7 MiB/s
INFO: 74% (95.2 GiB of 128.0 GiB) in 1m 19s, read: 6.9 GiB/s, write: 18.7 MiB/s
INFO: 76% (98.2 GiB of 128.0 GiB) in 1m 22s, read: 1006.7 MiB/s, write: 14.7 MiB/s
INFO: 91% (117.1 GiB of 128.0 GiB) in 1m 25s, read: 6.3 GiB/s, write: 0 B/s
INFO: 100% (128.0 GiB of 128.0 GiB) in 1m 28s, read: 3.6 GiB/s, write: 0 B/s
INFO: Waiting for server to finish backup validation...
INFO: backup is sparse: 113.46 GiB (88%) total zero data
INFO: backup was done incrementally, reused 126.40 GiB (98%)
INFO: transferred 128.00 GiB in 90 seconds (1.4 GiB/s)
INFO: adding notes to backup
INFO: prune older backups with retention: keep-last=14, keep-monthly=6
INFO: running 'proxmox-backup-client prune' for 'vm/1017'
INFO: pruned 1 backup(s) not covered by keep-retention policy
INFO: Finished Backup of VM 1017 (00:01:31)
INFO: Backup finished at 2025-11-12 11:21:51
INFO: Starting Backup of VM 1018 (qemu)
INFO: Backup started at 2025-11-12 11:21:51
INFO: status = running
INFO: VM Name: HDC-BM-MAAS01
INFO: include disk 'scsi0' 'Clusterdata:1018/vm-1018-disk-1.qcow2' 128G
INFO: include disk 'efidisk0' 'Clusterdata:1018/vm-1018-disk-0.qcow2' 528K
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating Proxmox Backup Server archive 'vm/1018/2025-11-12T10:21:51Z'
INFO: issuing guest-agent 'fs-freeze' command
INFO: issuing guest-agent 'fs-thaw' command
INFO: started backup task 'b58026cf-1f57-463d-ae76-da248763793d'
INFO: resuming VM again
INFO: efidisk0: dirty-bitmap status: OK (drive clean)
INFO: scsi0: dirty-bitmap status: OK (1.2 GiB of 128.0 GiB dirty)
INFO: using fast incremental mode (dirty-bitmap), 1.2 GiB dirty of 128.0 GiB total
INFO: 34% (408.0 MiB of 1.2 GiB) in 3s, read: 136.0 MiB/s, write: 134.7 MiB/s
ERROR: interrupted by signal
INFO: aborting backup job
INFO: resuming VM again
ERROR: Backup of VM 1018 failed - interrupted by signal
INFO: Failed at 2025-11-12 11:32:29
ERROR: Backup job failed - interrupted by signal
INFO: notified via target `<>`
TASK ERROR: interrupted by signal
The following has already been tried without result:
-restart hosts
-delete back-ups jobs and make them again
-delete PBS storage and add it again under a different name
-restart PBS
Can somebody tell what is causing this and how we can solve this problem?
With regards,
Jan Straatman
INFO: starting new backup job: vzdump 1001 1002 1017 1018 1012 --fleecing 0 --node have-hst03 --all 0 --prune-backups 'keep-last=14,keep-monthly=6' --storage pmbs-hdc --mailnotification failure --notes-template '{{guestname}}' --mode snapshot --mailto
INFO: Starting Backup of VM 1012 (qemu)
INFO: Backup started at 2025-11-12 11:20:11
INFO: status = running
INFO: VM Name: HDC-LNMS01-DEB12
INFO: include disk 'scsi0' 'Clusterdata:1012/vm-1012-disk-1.qcow2' 32G
INFO: include disk 'efidisk0' 'Clusterdata:1012/vm-1012-disk-0.qcow2' 528K
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating Proxmox Backup Server archive 'vm/1012/2025-11-12T10:20:11Z'
INFO: issuing guest-agent 'fs-freeze' command
INFO: issuing guest-agent 'fs-thaw' command
INFO: started backup task 'b9219771-2526-41aa-a3cc-fd3b7eba3746'
INFO: resuming VM again
INFO: efidisk0: dirty-bitmap status: OK (drive clean)
INFO: scsi0: dirty-bitmap status: OK (672.0 MiB of 32.0 GiB dirty)
INFO: using fast incremental mode (dirty-bitmap), 672.0 MiB dirty of 32.0 GiB total
INFO: 100% (672.0 MiB of 672.0 MiB) in 3s, read: 224.0 MiB/s, write: 218.7 MiB/s
INFO: backup was done incrementally, reused 31.36 GiB (97%)
INFO: transferred 672.00 MiB in 7 seconds (96.0 MiB/s)
INFO: adding notes to backup
INFO: prune older backups with retention: keep-last=14, keep-monthly=6
INFO: running 'proxmox-backup-client prune' for 'vm/1012'
INFO: pruned 1 backup(s) not covered by keep-retention policy
INFO: Finished Backup of VM 1012 (00:00:09)
INFO: Backup finished at 2025-11-12 11:20:20
INFO: Starting Backup of VM 1017 (qemu)
INFO: Backup started at 2025-11-12 11:20:20
INFO: status = running
INFO: VM Name: HDC-WS01-DEB13
INFO: include disk 'scsi0' 'Clusterdata:1017/vm-1017-disk-1.qcow2' 128G
INFO: include disk 'efidisk0' 'Clusterdata:1017/vm-1017-disk-0.qcow2' 528K
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating Proxmox Backup Server archive 'vm/1017/2025-11-12T10:20:20Z'
INFO: issuing guest-agent 'fs-freeze' command
INFO: issuing guest-agent 'fs-thaw' command
INFO: started backup task 'fe37a194-8845-4988-9e83-11a4cc5ec0f5'
INFO: resuming VM again
INFO: efidisk0: dirty-bitmap status: OK (drive clean)
INFO: scsi0: dirty-bitmap status: existing bitmap was invalid and has been cleared
INFO: using fast incremental mode (dirty-bitmap), 128.0 GiB dirty of 128.0 GiB total
INFO: 1% (2.4 GiB of 128.0 GiB) in 3s, read: 816.0 MiB/s, write: 50.7 MiB/s
INFO: 3% (4.9 GiB of 128.0 GiB) in 6s, read: 846.7 MiB/s, write: 24.0 MiB/s
INFO: 4% (5.2 GiB of 128.0 GiB) in 10s, read: 74.0 MiB/s, write: 38.0 MiB/s
INFO: 5% (6.5 GiB of 128.0 GiB) in 52s, read: 31.6 MiB/s, write: 19.5 MiB/s
INFO: 6% (8.2 GiB of 128.0 GiB) in 1m 4s, read: 145.7 MiB/s, write: 1.3 MiB/s
INFO: 19% (24.4 GiB of 128.0 GiB) in 1m 7s, read: 5.4 GiB/s, write: 0 B/s
INFO: 32% (41.8 GiB of 128.0 GiB) in 1m 10s, read: 5.8 GiB/s, write: 6.7 MiB/s
INFO: 43% (56.0 GiB of 128.0 GiB) in 1m 13s, read: 4.7 GiB/s, write: 4.0 MiB/s
INFO: 58% (74.5 GiB of 128.0 GiB) in 1m 16s, read: 6.2 GiB/s, write: 98.7 MiB/s
INFO: 74% (95.2 GiB of 128.0 GiB) in 1m 19s, read: 6.9 GiB/s, write: 18.7 MiB/s
INFO: 76% (98.2 GiB of 128.0 GiB) in 1m 22s, read: 1006.7 MiB/s, write: 14.7 MiB/s
INFO: 91% (117.1 GiB of 128.0 GiB) in 1m 25s, read: 6.3 GiB/s, write: 0 B/s
INFO: 100% (128.0 GiB of 128.0 GiB) in 1m 28s, read: 3.6 GiB/s, write: 0 B/s
INFO: Waiting for server to finish backup validation...
INFO: backup is sparse: 113.46 GiB (88%) total zero data
INFO: backup was done incrementally, reused 126.40 GiB (98%)
INFO: transferred 128.00 GiB in 90 seconds (1.4 GiB/s)
INFO: adding notes to backup
INFO: prune older backups with retention: keep-last=14, keep-monthly=6
INFO: running 'proxmox-backup-client prune' for 'vm/1017'
INFO: pruned 1 backup(s) not covered by keep-retention policy
INFO: Finished Backup of VM 1017 (00:01:31)
INFO: Backup finished at 2025-11-12 11:21:51
INFO: Starting Backup of VM 1018 (qemu)
INFO: Backup started at 2025-11-12 11:21:51
INFO: status = running
INFO: VM Name: HDC-BM-MAAS01
INFO: include disk 'scsi0' 'Clusterdata:1018/vm-1018-disk-1.qcow2' 128G
INFO: include disk 'efidisk0' 'Clusterdata:1018/vm-1018-disk-0.qcow2' 528K
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating Proxmox Backup Server archive 'vm/1018/2025-11-12T10:21:51Z'
INFO: issuing guest-agent 'fs-freeze' command
INFO: issuing guest-agent 'fs-thaw' command
INFO: started backup task 'b58026cf-1f57-463d-ae76-da248763793d'
INFO: resuming VM again
INFO: efidisk0: dirty-bitmap status: OK (drive clean)
INFO: scsi0: dirty-bitmap status: OK (1.2 GiB of 128.0 GiB dirty)
INFO: using fast incremental mode (dirty-bitmap), 1.2 GiB dirty of 128.0 GiB total
INFO: 34% (408.0 MiB of 1.2 GiB) in 3s, read: 136.0 MiB/s, write: 134.7 MiB/s
ERROR: interrupted by signal
INFO: aborting backup job
INFO: resuming VM again
ERROR: Backup of VM 1018 failed - interrupted by signal
INFO: Failed at 2025-11-12 11:32:29
ERROR: Backup job failed - interrupted by signal
INFO: notified via target `<>`
TASK ERROR: interrupted by signal
The following has already been tried without result:
-restart hosts
-delete back-ups jobs and make them again
-delete PBS storage and add it again under a different name
-restart PBS
Can somebody tell what is causing this and how we can solve this problem?
With regards,
Jan Straatman
