I use PBS to backup various VMs / CTs to a SAMBA share on a NAS.
I've set the NAS to automatically turn on 20 mins before the backup job, then off again about an hour after it should finish, using its own power schedule. (This is to save power, and also prolong disk life.)
The backup works, but the following night, the backup fails within a second of starting, even though the NAS is on, and the datastore can be browsed from within PBS (i.e. so it's connected fine).
If I reboot PBS and run the backup manually, it works fine.
Why could this be?
I'll paste error text for the job if it helps?
I've set the NAS to automatically turn on 20 mins before the backup job, then off again about an hour after it should finish, using its own power schedule. (This is to save power, and also prolong disk life.)
The backup works, but the following night, the backup fails within a second of starting, even though the NAS is on, and the datastore can be browsed from within PBS (i.e. so it's connected fine).
If I reboot PBS and run the backup manually, it works fine.
Why could this be?
I'll paste error text for the job if it helps?
Code:
INFO: starting new backup job: vzdump 101 103 104 105 100 109 --mode snapshot --prune-backups 'keep-daily=5,keep-last=15,keep-monthly=5,keep-yearly=2' --all 0 --node pm --fleecing 0 --storage pbs --notes-template '{{guestname}}'
INFO: Starting Backup of VM 100 (qemu)
INFO: Backup started at 2025-02-10 21:18:10
INFO: status = running
INFO: VM Name: prebend-cctv
INFO: include disk 'scsi1' 'local-lvm:vm-100-disk-1' 127G
INFO: exclude disk 'scsi2' 'nvme-vol-group:vm-100-disk-0' (backup=no)
INFO: include disk 'efidisk0' 'local-lvm:vm-100-disk-0' 4M
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: skip unused drive 'local-lvm:vm-100-disk-2' (not included into backup)
INFO: creating Proxmox Backup Server archive 'vm/100/2025-02-10T21:18:10Z'
INFO: issuing guest-agent 'fs-freeze' command
INFO: issuing guest-agent 'fs-thaw' command
ERROR: VM 100 qmp command 'backup' failed - backup register image failed: command error: stream error received: stream no longer needed
INFO: aborting backup job
INFO: resuming VM again
ERROR: Backup of VM 100 failed - VM 100 qmp command 'backup' failed - backup register image failed: command error: stream error received: stream no longer needed
INFO: Failed at 2025-02-10 21:18:13
INFO: Starting Backup of VM 101 (qemu)
INFO: Backup started at 2025-02-10 21:18:13
INFO: status = running
INFO: VM Name: automate
INFO: include disk 'scsi0' 'local-lvm:vm-101-disk-0' 127G
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: snapshots found (not included into backup)
INFO: creating Proxmox Backup Server archive 'vm/101/2025-02-10T21:18:13Z'
INFO: issuing guest-agent 'fs-freeze' command
INFO: issuing guest-agent 'fs-thaw' command
ERROR: VM 101 qmp command 'backup' failed - backup register image failed: command error: stream error received: stream no longer needed
INFO: aborting backup job
INFO: resuming VM again
ERROR: Backup of VM 101 failed - VM 101 qmp command 'backup' failed - backup register image failed: command error: stream error received: stream no longer needed
INFO: Failed at 2025-02-10 21:18:14
INFO: Starting Backup of VM 103 (qemu)
INFO: Backup started at 2025-02-10 21:18:14
INFO: status = running
INFO: VM Name: HomeAssistant
INFO: include disk 'scsi0' 'local-lvm:vm-103-disk-0' 32G
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: snapshots found (not included into backup)
INFO: creating Proxmox Backup Server archive 'vm/103/2025-02-10T21:18:14Z'
ERROR: VM 103 qmp command 'backup' failed - backup register image failed: command error: stream error received: stream no longer needed
INFO: aborting backup job
INFO: resuming VM again
ERROR: Backup of VM 103 failed - VM 103 qmp command 'backup' failed - backup register image failed: command error: stream error received: stream no longer needed
INFO: Failed at 2025-02-10 21:18:14
INFO: Starting Backup of VM 104 (qemu)
INFO: Backup started at 2025-02-10 21:18:14
INFO: status = running
INFO: VM Name: Z2MQTT
INFO: include disk 'scsi0' 'local-lvm:vm-104-disk-0' 127G
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating Proxmox Backup Server archive 'vm/104/2025-02-10T21:18:14Z'
ERROR: VM 104 qmp command 'backup' failed - backup register image failed: command error: stream error received: stream no longer needed
INFO: aborting backup job
INFO: resuming VM again
ERROR: Backup of VM 104 failed - VM 104 qmp command 'backup' failed - backup register image failed: command error: stream error received: stream no longer needed
INFO: Failed at 2025-02-10 21:18:14
INFO: Starting Backup of VM 109 (lxc)
INFO: Backup started at 2025-02-10 21:18:14
INFO: status = running
INFO: CT Name: whitecity-media
INFO: including mount point rootfs ('/') in backup
INFO: excluding volume mount point mp0 ('/mnt/media') from backup (disabled)
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: create storage snapshot 'vzdump'
Logical volume "snap_vm-109-disk-0_vzdump" created.
INFO: creating Proxmox Backup Server archive 'ct/109/2025-02-10T21:18:14Z'
INFO: set max number of entries in memory for file-based backups to 1048576
INFO: run: lxc-usernsexec -m u:0:100000:65536 -m g:0:100000:65536 -- /usr/bin/proxmox-backup-client backup --crypt-mode=none pct.conf:/var/tmp/vzdumptmp3720642_109/etc/vzdump/pct.conf root.pxar:/mnt/vzsnap0 --include-dev /mnt/vzsnap0/./ --skip-lost-and-found --exclude=/tmp/?* --exclude=/var/tmp/?* --exclude=/var/run/?*.pid --backup-type ct --backup-id 109 --backup-time 1739222294 --entries-max 1048576 --repository root@pam@10.1.1.32:pbs
INFO: Starting backup: ct/109/2025-02-10T21:18:14Z
INFO: Client name: pm
INFO: Starting backup protocol: Mon Feb 10 21:18:15 2025
INFO: Downloading previous manifest (Sun Feb 9 21:21:41 2025)
INFO: Upload config file '/var/tmp/vzdumptmp3720642_109/etc/vzdump/pct.conf' to 'root@pam@10.1.1.32:8007:pbs' as pct.conf.blob
INFO: Upload directory '/mnt/vzsnap0' to 'root@pam@10.1.1.32:8007:pbs' as root.pxar.didx
INFO: catalog upload error - stream error received: stream no longer needed
INFO: Error downloading .didx from previous manifest: stream closed because of a broken pipe
INFO: unclosed encoder dropped
INFO: closed encoder dropped with state
INFO: unfinished encoder state dropped
INFO: Error: connection closed because of a broken pipe
INFO: cleanup temporary 'vzdump' snapshot
Logical volume "snap_vm-109-disk-0_vzdump" successfully removed.
ERROR: Backup of VM 109 failed - command 'lxc-usernsexec -m u:0:100000:65536 -m g:0:100000:65536 -- /usr/bin/proxmox-backup-client backup '--crypt-mode=none' pct.conf:/var/tmp/vzdumptmp3720642_109/etc/vzdump/pct.conf root.pxar:/mnt/vzsnap0 --include-dev /mnt/vzsnap0/./ --skip-lost-and-found '--exclude=/tmp/?*' '--exclude=/var/tmp/?*' '--exclude=/var/run/?*.pid' --backup-type ct --backup-id 109 --backup-time 1739222294 --entries-max 1048576 --repository root@pam@10.1.1.32:pbs' failed: exit code 255
INFO: Failed at 2025-02-10 21:18:15
INFO: Backup job finished with errors
TASK ERROR: job errors