Error when backing up 1 out of 4 vms only.timesout

M

mason64

Member
Apr 18, 2022
39
5
13
Hi all

I have been backing up 4 vms from a Proxmox server to a proxmox backup server for months now with no errors, over the last few weeks 4 or 5 times a week i get an error (See the log below) only on 1 out of the 4 vms that i backup, i run two backups a day sometimes both error sometimes only 1 and sometimes the backup completes. If i run a manually backup it works every time.


Code: 
201: 2026-04-09 22:30:07 INFO: Starting Backup of VM 201 (qemu)
201: 2026-04-09 22:30:07 INFO: status = running
201: 2026-04-09 22:30:07 INFO: VM Name: lwportal-web-server
201: 2026-04-09 22:30:07 INFO: include disk 'sata0' 'VMStorage:vm-201-disk-0' 1000G
201: 2026-04-09 22:30:07 INFO: backup mode: snapshot
201: 2026-04-09 22:30:07 INFO: ionice priority: 7
201: 2026-04-09 22:30:07 INFO: creating Proxmox Backup Server archive 'vm/201/2026-04-09T21:30:07Z'
201: 2026-04-09 22:30:07 INFO: issuing guest-agent 'fs-freeze' command
201: 2026-04-09 22:30:08 INFO: issuing guest-agent 'fs-thaw' command
201: 2026-04-09 22:30:08 INFO: started backup task '4c98baae-0b8f-4c5d-992a-8ba09629eeff'
201: 2026-04-09 22:30:08 INFO: resuming VM again
201: 2026-04-09 22:30:08 INFO: sata0: dirty-bitmap status: OK (28.1 GiB of 1000.0 GiB dirty)
201: 2026-04-09 22:30:08 INFO: using fast incremental mode (dirty-bitmap), 28.1 GiB dirty of 1000.0 GiB total
201: 2026-04-09 22:30:11 INFO:   2% (700.0 MiB of 28.1 GiB) in 3s, read: 233.3 MiB/s, write: 216.0 MiB/s
201: 2026-04-09 22:30:14 INFO:   4% (1.3 GiB of 28.1 GiB) in 6s, read: 217.3 MiB/s, write: 217.3 MiB/s
201: 2026-04-09 22:30:17 INFO:   7% (2.0 GiB of 28.1 GiB) in 9s, read: 228.0 MiB/s, write: 228.0 MiB/s
201: 2026-04-09 22:30:20 INFO:   9% (2.6 GiB of 28.1 GiB) in 12s, read: 222.7 MiB/s, write: 210.7 MiB/s
201: 2026-04-09 22:30:23 INFO:  11% (3.3 GiB of 28.1 GiB) in 15s, read: 225.3 MiB/s, write: 218.7 MiB/s
201: 2026-04-09 22:30:26 INFO:  14% (4.1 GiB of 28.1 GiB) in 18s, read: 277.3 MiB/s, write: 153.3 MiB/s
201: 2026-04-09 22:30:29 INFO:  17% (4.9 GiB of 28.1 GiB) in 21s, read: 257.3 MiB/s, write: 172.0 MiB/s
201: 2026-04-09 22:30:32 INFO:  20% (5.8 GiB of 28.1 GiB) in 24s, read: 316.0 MiB/s, write: 85.3 MiB/s
201: 2026-04-09 22:30:35 INFO:  23% (6.6 GiB of 28.1 GiB) in 27s, read: 264.0 MiB/s, write: 113.3 MiB/s
201: 2026-04-09 22:30:38 INFO:  26% (7.3 GiB of 28.1 GiB) in 30s, read: 257.3 MiB/s, write: 109.3 MiB/s
201: 2026-04-09 22:30:41 INFO:  27% (7.8 GiB of 28.1 GiB) in 33s, read: 153.3 MiB/s, write: 153.3 MiB/s
201: 2026-04-09 22:30:44 INFO:  29% (8.2 GiB of 28.1 GiB) in 36s, read: 134.7 MiB/s, write: 134.7 MiB/s
201: 2026-04-09 22:30:47 INFO:  30% (8.5 GiB of 28.1 GiB) in 39s, read: 120.0 MiB/s, write: 120.0 MiB/s
201: 2026-04-09 22:30:50 INFO:  31% (8.8 GiB of 28.1 GiB) in 42s, read: 112.0 MiB/s, write: 112.0 MiB/s
201: 2026-04-09 22:30:53 INFO:  32% (9.2 GiB of 28.1 GiB) in 45s, read: 116.0 MiB/s, write: 116.0 MiB/s
201: 2026-04-09 22:30:56 INFO:  34% (9.7 GiB of 28.1 GiB) in 48s, read: 176.0 MiB/s, write: 170.7 MiB/s
201: 2026-04-09 22:30:59 INFO:  36% (10.3 GiB of 28.1 GiB) in 51s, read: 216.0 MiB/s, write: 210.7 MiB/s
201: 2026-04-09 22:31:02 INFO:  38% (10.9 GiB of 28.1 GiB) in 54s, read: 198.7 MiB/s, write: 198.7 MiB/s
201: 2026-04-09 22:31:05 INFO:  40% (11.5 GiB of 28.1 GiB) in 57s, read: 197.3 MiB/s, write: 197.3 MiB/s
201: 2026-04-09 22:31:08 INFO:  43% (12.3 GiB of 28.1 GiB) in 1m, read: 268.0 MiB/s, write: 160.0 MiB/s
201: 2026-04-09 22:31:11 INFO:  46% (13.0 GiB of 28.1 GiB) in 1m 3s, read: 237.3 MiB/s, write: 26.7 MiB/s
201: 2026-04-09 22:41:46 ERROR: VM 201 qmp command 'query-backup' failed - got timeout
201: 2026-04-09 22:41:46 INFO: aborting backup job
201: 2026-04-09 22:47:01 INFO: resuming VM again
201: 2026-04-09 22:47:01 ERROR: Backup of VM 201 failed - VM 201 qmp command 'query-backup' failed - got timeout


Any reason why it would get timed out? Anything i can do to fix this. The vm is running a web server and when it times out the web server also is not accessable for that time out time.
Thank you.
 
Last edited:
mason64 said:
Any reason why it would get timed out? Anything i can do to fix this. The vm is running a web server and when it times out the web server also is not accessable for that time out time.
Click to expand...
Please check the backup task log and systemd journal on the PBS for errors. Also, do you see high IO delay or load on the PBS during backup failures? Any other IO heavy tasks such as verification or garbage collection which run in parallel? Further, in order to decouple VM I/O from the backup task, using backup fleecing is recommended, see https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_vm_backup_fleecing
 
Hi @Chris

Thanks for the quick reply, Can i just check where do i look for the backup task log? and look for IO delays do i check that on the PBS or PVE server?

Thanks
 
mason64 said:
Hi @Chris

Thanks for the quick reply, Can i just check where do i look for the backup task log? and look for IO delays do i check that on the PBS or PVE server?
Click to expand...
You should check the IO delay and load on the PBS, the task logs you can find under Administration > Tasks
 
Thanks @Chris for the reply. The task logs say this when it fails

Code: 
2026-04-09T22:30:07+01:00: starting new backup on datastore 'id0-disk7-8' from ::ffff:192.168.1.9: "vm/201/2026-04-09T21:30:07Z"
2026-04-09T22:30:07+01:00: download 'index.json.blob' from previous backup 'vm/201/2026-04-09T01:30:09Z'.
2026-04-09T22:30:07+01:00: register chunks in 'drive-sata0.img.fidx' from previous backup 'vm/201/2026-04-09T01:30:09Z'.
2026-04-09T22:30:07+01:00: download 'drive-sata0.img.fidx' from previous backup 'vm/201/2026-04-09T01:30:09Z'.
2026-04-09T22:30:08+01:00: created new fixed index 1 ("vm/201/2026-04-09T21:30:07Z/drive-sata0.img.fidx")
2026-04-09T22:30:08+01:00: add blob "/mnt/datastore/id0-disk7-8/vm/201/2026-04-09T21:30:07Z/qemu-server.conf.blob" (404 bytes, comp: 404)
2026-04-09T22:47:30+01:00: backup failed: connection error
2026-04-09T22:47:30+01:00: removing failed backup
2026-04-09T22:47:30+01:00: removing backup snapshot "/mnt/datastore/id0-disk7-8/vm/201/2026-04-09T21:30:07Z"
2026-04-09T22:47:30+01:00: POST /fixed_chunk: 400 Bad Request: error reading a body from connection
2026-04-09T22:47:30+01:00: TASK ERROR: connection error: connection reset
 
You must log in or register to reply here.
Top Bottom
Back