Backup failure (repeatedly)

[leksand]

Recently, there have been regular (once a few days, sometimes once a day) backup failures on proxmox backup server.

The log is below, in general, the machine is locked and has not been unblocked after. Subsequent viral machines are also all failed (Permission denied).

What can I do, do I need additional information or logs?

pve 8.0.3
pbs 3.0-1

190: 2023-08-16 02:18:52 INFO: Starting Backup of VM 190 (qemu)
190: 2023-08-16 02:18:52 INFO: status = running
190: 2023-08-16 02:18:52 INFO: VM Name: ipa1
190: 2023-08-16 02:18:52 INFO: include disk 'scsi0' 'pve3-nvme:vm-190-disk-0' 10G
190: 2023-08-16 02:18:52 INFO: backup mode: snapshot
190: 2023-08-16 02:18:52 INFO: ionice priority: 7
190: 2023-08-16 02:18:52 INFO: creating Proxmox Backup Server archive 'vm/190/2023-08-15T23:18:52Z'
190: 2023-08-16 02:18:52 INFO: issuing guest-agent 'fs-freeze' command
190: 2023-08-16 02:18:54 INFO: issuing guest-agent 'fs-thaw' command
190: 2023-08-16 02:18:55 INFO: started backup task '6a348630-1d38-4940-8829-05266dc09b32'
190: 2023-08-16 02:18:55 INFO: resuming VM again
190: 2023-08-16 02:18:55 INFO: scsi0: dirty-bitmap status: OK (848.0 MiB of 10.0 GiB dirty)
190: 2023-08-16 02:18:55 INFO: using fast incremental mode (dirty-bitmap), 848.0 MiB dirty of 10.0 GiB total
190: 2023-08-16 02:18:58 INFO: 21% (184.0 MiB of 848.0 MiB) in 3s, read: 61.3 MiB/s, write: 61.3 MiB/s
190: 2023-08-16 02:19:01 INFO: 40% (344.0 MiB of 848.0 MiB) in 6s, read: 53.3 MiB/s, write: 53.3 MiB/s
190: 2023-08-16 02:19:04 INFO: 68% (584.0 MiB of 848.0 MiB) in 9s, read: 80.0 MiB/s, write: 78.7 MiB/s
190: 2023-08-16 02:19:07 INFO: 69% (592.0 MiB of 848.0 MiB) in 12s, read: 2.7 MiB/s, write: 2.7 MiB/s
190: 2023-08-16 02:19:59 INFO: 69% (592.0 MiB of 848.0 MiB) in 1m 4s, read: 0 B/s, write: 0 B/s
190: 2023-08-16 02:19:59 ERROR: backup write data failed: command error: protocol canceled
190: 2023-08-16 02:19:59 INFO: aborting backup job
190: 2023-08-16 02:19:59 INFO: resuming VM again
190: 2023-08-16 02:19:59 ERROR: Backup of VM 190 failed - backup write data failed: command error: protocol canceled

198: 2023-08-16 02:19:59 INFO: Starting Backup of VM 198 (qemu)
198: 2023-08-16 02:19:59 INFO: status = running
198: 2023-08-16 02:19:59 ERROR: Backup of VM 198 failed - unable to open file '/etc/pve/nodes/pve3/qemu-server/198.conf.tmp.2906895' - Permission denied

 

[Chris]

Hi, please append the journal from around the time the error happens from the PBS host and the PVE host, journalctl --since <DATETIME> --until <DATETIME> > journal.txt.