This could be just "one of those things" but thought I'd report as it is a PBS that was recently upgraded to version 4.0.14. Since the upgrade, it has been running fine. I've done test restores and it appears to be OK.
Last night I got the following log:
INFO: starting new backup job: vzdump --all 1 --fleecing 0 --mailnotification always --notes-template '{{guestname}}' --mailto root@pam --mode snapshot --notification-mode notification-system --quiet 1 --storage PBS1
INFO: skip external VMs: 100, 105, 110, 150, 201, 202, 203, 206, 210, 211, 301
INFO: Starting Backup of VM 300 (qemu)
INFO: Backup started at 2025-08-26 20:00:04
INFO: status = running
INFO: VM Name: PR-FILE1
INFO: include disk 'virtio0' 'Datastore:vm-300-disk-0' 128G
INFO: include disk 'virtio1' 'Datastore:vm-300-disk-1' 300G
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: snapshots found (not included into backup)
INFO: creating Proxmox Backup Server archive 'vm/300/2025-08-27T00:00:04Z'
INFO: issuing guest-agent 'fs-freeze' command
INFO: issuing guest-agent 'fs-thaw' command
INFO: started backup task 'e571840e-d3a8-4de3-952c-e32e5f85183f'
INFO: resuming VM again
INFO: virtio0: dirty-bitmap status: existing bitmap was invalid and has been cleared
INFO: virtio1: dirty-bitmap status: existing bitmap was invalid and has been cleared
INFO: 8% (35.6 GiB of 428.0 GiB) in 3s, read: 11.9 GiB/s, write: 129.3 MiB/s
INFO: 9% (40.6 GiB of 428.0 GiB) in 6s, read: 1.7 GiB/s, write: 57.3 MiB/s
INFO: 11% (47.2 GiB of 428.0 GiB) in 9s, read: 2.2 GiB/s, write: 32.0 MiB/s
...
INFO: 99% (426.4 GiB of 428.0 GiB) in 3m 22s, read: 1.5 GiB/s, write: 160.0 MiB/s
INFO: 100% (428.0 GiB of 428.0 GiB) in 3m 25s, read: 545.3 MiB/s, write: 5.3 MiB/s
INFO: backup is sparse: 171.98 GiB (40%) total zero data
INFO: backup was done incrementally, reused 398.76 GiB (93%)
INFO: transferred 428.00 GiB in 205 seconds (2.1 GiB/s)
INFO: adding notes to backup
INFO: Finished Backup of VM 300 (00:03:32)
INFO: Backup finished at 2025-08-26 20:03:36
INFO: Starting Backup of VM 302 (qemu)
INFO: Backup started at 2025-08-26 20:03:36
INFO: status = running
INFO: VM Name: PR-APPS2
INFO: include disk 'virtio0' 'Datastore:vm-302-disk-0' 170G
INFO: include disk 'virtio1' 'Datastore:vm-302-disk-1' 450G
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: snapshots found (not included into backup)
INFO: creating Proxmox Backup Server archive 'vm/302/2025-08-27T00:03:36Z'
INFO: issuing guest-agent 'fs-freeze' command
INFO: issuing guest-agent 'fs-thaw' command
INFO: started backup task '7eedd0d2-ccb5-4a45-aa7b-4b73717bb086'
INFO: resuming VM again
INFO: virtio0: dirty-bitmap status: existing bitmap was invalid and has been cleared
INFO: virtio1: dirty-bitmap status: existing bitmap was invalid and has been cleared
INFO: 1% (7.2 GiB of 620.0 GiB) in 3s, read: 2.4 GiB/s, write: 257.3 MiB/s
INFO: 2% (14.0 GiB of 620.0 GiB) in 7s, read: 1.7 GiB/s, write: 21.0 MiB/s
...
INFO: 99% (614.5 GiB of 620.0 GiB) in 6m 28s, read: 1.6 GiB/s, write: 159.0 MiB/s
INFO: 100% (620.0 GiB of 620.0 GiB) in 6m 31s, read: 1.8 GiB/s, write: 100.0 MiB/s
INFO: Waiting for server to finish backup validation...
INFO: backup is sparse: 33.56 GiB (5%) total zero data
INFO: backup was done incrementally, reused 582.93 GiB (94%)
INFO: transferred 620.00 GiB in 393 seconds (1.6 GiB/s)
INFO: adding notes to backup
INFO: Finished Backup of VM 302 (00:06:39)
INFO: Backup finished at 2025-08-26 20:10:15
INFO: Starting Backup of VM 303 (qemu)
INFO: Backup started at 2025-08-26 20:10:16
INFO: status = running
INFO: VM Name: PR-RDS2
INFO: include disk 'virtio0' 'Datastore:vm-303-disk-0' 210G
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: snapshots found (not included into backup)
INFO: creating Proxmox Backup Server archive 'vm/303/2025-08-27T00:10:16Z'
INFO: issuing guest-agent 'fs-freeze' command
INFO: issuing guest-agent 'fs-thaw' command
ERROR: VM 303 qmp command 'backup' failed - backup register image failed: command error: error:0A00010B:SSL routines:ssl3_get_record:wrong version number:../ssl/record/ssl3_record.c:358:
INFO: aborting backup job
INFO: resuming VM again
ERROR: Backup of VM 303 failed - VM 303 qmp command 'backup' failed - backup register image failed: command error: error:0A00010B:SSL routines:ssl3_get_record:wrong version number:../ssl/record/ssl3_record.c:358:
INFO: Failed at 2025-08-26 20:10:20
INFO: Backup job finished with errors
INFO: notified via target `Lee`
TASK ERROR: job errors
So the last backup failed on VM 303. It's unclear if this is a PBS error or a PVE error. The details of servers used are:
PBS1: On version 4.0.14. Non-enterprise. Bare metal PBS on HP Microserver Gen 8. Xeon E3-1265L. 16GB Ram. One boot SSD and 4 SSD set up as ZFS RAID 10. All SSD are enterprise grade Samsung SM863a. 10Gbe Mellanox NIC.
PVE3: On versoin 8.4.11. Enterprise license. Part of a 3 node cluster. On HP Microserver Gen 11. Xeon E-2468. 128GB Ram. One boot NVME and 4 SSD set up as ZFS RAID 10. All SSD are enterprise grade Samsung SM863a. 10Gbe Mellanox NIC.
All other VMs on both this node and other nodes were backed up without error. When I logged in this morning, I was able to run the backup for the failed VM. And the VM was up and running without any problems. Our other PBS is on versoin 3.4.6 and runs with an enterprise license and had no errors. No backup times conflict.
I don't need any help with this. It could be just "one of those things". But where PBS 4 is so new, I thought I'd report it on an abundance of caution.
Last night I got the following log:
INFO: starting new backup job: vzdump --all 1 --fleecing 0 --mailnotification always --notes-template '{{guestname}}' --mailto root@pam --mode snapshot --notification-mode notification-system --quiet 1 --storage PBS1
INFO: skip external VMs: 100, 105, 110, 150, 201, 202, 203, 206, 210, 211, 301
INFO: Starting Backup of VM 300 (qemu)
INFO: Backup started at 2025-08-26 20:00:04
INFO: status = running
INFO: VM Name: PR-FILE1
INFO: include disk 'virtio0' 'Datastore:vm-300-disk-0' 128G
INFO: include disk 'virtio1' 'Datastore:vm-300-disk-1' 300G
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: snapshots found (not included into backup)
INFO: creating Proxmox Backup Server archive 'vm/300/2025-08-27T00:00:04Z'
INFO: issuing guest-agent 'fs-freeze' command
INFO: issuing guest-agent 'fs-thaw' command
INFO: started backup task 'e571840e-d3a8-4de3-952c-e32e5f85183f'
INFO: resuming VM again
INFO: virtio0: dirty-bitmap status: existing bitmap was invalid and has been cleared
INFO: virtio1: dirty-bitmap status: existing bitmap was invalid and has been cleared
INFO: 8% (35.6 GiB of 428.0 GiB) in 3s, read: 11.9 GiB/s, write: 129.3 MiB/s
INFO: 9% (40.6 GiB of 428.0 GiB) in 6s, read: 1.7 GiB/s, write: 57.3 MiB/s
INFO: 11% (47.2 GiB of 428.0 GiB) in 9s, read: 2.2 GiB/s, write: 32.0 MiB/s
...
INFO: 99% (426.4 GiB of 428.0 GiB) in 3m 22s, read: 1.5 GiB/s, write: 160.0 MiB/s
INFO: 100% (428.0 GiB of 428.0 GiB) in 3m 25s, read: 545.3 MiB/s, write: 5.3 MiB/s
INFO: backup is sparse: 171.98 GiB (40%) total zero data
INFO: backup was done incrementally, reused 398.76 GiB (93%)
INFO: transferred 428.00 GiB in 205 seconds (2.1 GiB/s)
INFO: adding notes to backup
INFO: Finished Backup of VM 300 (00:03:32)
INFO: Backup finished at 2025-08-26 20:03:36
INFO: Starting Backup of VM 302 (qemu)
INFO: Backup started at 2025-08-26 20:03:36
INFO: status = running
INFO: VM Name: PR-APPS2
INFO: include disk 'virtio0' 'Datastore:vm-302-disk-0' 170G
INFO: include disk 'virtio1' 'Datastore:vm-302-disk-1' 450G
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: snapshots found (not included into backup)
INFO: creating Proxmox Backup Server archive 'vm/302/2025-08-27T00:03:36Z'
INFO: issuing guest-agent 'fs-freeze' command
INFO: issuing guest-agent 'fs-thaw' command
INFO: started backup task '7eedd0d2-ccb5-4a45-aa7b-4b73717bb086'
INFO: resuming VM again
INFO: virtio0: dirty-bitmap status: existing bitmap was invalid and has been cleared
INFO: virtio1: dirty-bitmap status: existing bitmap was invalid and has been cleared
INFO: 1% (7.2 GiB of 620.0 GiB) in 3s, read: 2.4 GiB/s, write: 257.3 MiB/s
INFO: 2% (14.0 GiB of 620.0 GiB) in 7s, read: 1.7 GiB/s, write: 21.0 MiB/s
...
INFO: 99% (614.5 GiB of 620.0 GiB) in 6m 28s, read: 1.6 GiB/s, write: 159.0 MiB/s
INFO: 100% (620.0 GiB of 620.0 GiB) in 6m 31s, read: 1.8 GiB/s, write: 100.0 MiB/s
INFO: Waiting for server to finish backup validation...
INFO: backup is sparse: 33.56 GiB (5%) total zero data
INFO: backup was done incrementally, reused 582.93 GiB (94%)
INFO: transferred 620.00 GiB in 393 seconds (1.6 GiB/s)
INFO: adding notes to backup
INFO: Finished Backup of VM 302 (00:06:39)
INFO: Backup finished at 2025-08-26 20:10:15
INFO: Starting Backup of VM 303 (qemu)
INFO: Backup started at 2025-08-26 20:10:16
INFO: status = running
INFO: VM Name: PR-RDS2
INFO: include disk 'virtio0' 'Datastore:vm-303-disk-0' 210G
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: snapshots found (not included into backup)
INFO: creating Proxmox Backup Server archive 'vm/303/2025-08-27T00:10:16Z'
INFO: issuing guest-agent 'fs-freeze' command
INFO: issuing guest-agent 'fs-thaw' command
ERROR: VM 303 qmp command 'backup' failed - backup register image failed: command error: error:0A00010B:SSL routines:ssl3_get_record:wrong version number:../ssl/record/ssl3_record.c:358:
INFO: aborting backup job
INFO: resuming VM again
ERROR: Backup of VM 303 failed - VM 303 qmp command 'backup' failed - backup register image failed: command error: error:0A00010B:SSL routines:ssl3_get_record:wrong version number:../ssl/record/ssl3_record.c:358:
INFO: Failed at 2025-08-26 20:10:20
INFO: Backup job finished with errors
INFO: notified via target `Lee`
TASK ERROR: job errors
So the last backup failed on VM 303. It's unclear if this is a PBS error or a PVE error. The details of servers used are:
PBS1: On version 4.0.14. Non-enterprise. Bare metal PBS on HP Microserver Gen 8. Xeon E3-1265L. 16GB Ram. One boot SSD and 4 SSD set up as ZFS RAID 10. All SSD are enterprise grade Samsung SM863a. 10Gbe Mellanox NIC.
PVE3: On versoin 8.4.11. Enterprise license. Part of a 3 node cluster. On HP Microserver Gen 11. Xeon E-2468. 128GB Ram. One boot NVME and 4 SSD set up as ZFS RAID 10. All SSD are enterprise grade Samsung SM863a. 10Gbe Mellanox NIC.
All other VMs on both this node and other nodes were backed up without error. When I logged in this morning, I was able to run the backup for the failed VM. And the VM was up and running without any problems. Our other PBS is on versoin 3.4.6 and runs with an enterprise license and had no errors. No backup times conflict.
I don't need any help with this. It could be just "one of those things". But where PBS 4 is so new, I thought I'd report it on an abundance of caution.