The version of pve-qemu is 8.2.2-1; it has been found that this version causes timeouts of the QMP command for local backups of virtual machines, leading to backup failures and virtual machine hangs.
the backup logs:
INFO: starting new backup job: vzdump 100102002 --remove 0 --compress zstd --notes-template '{{guestname}}' --notification-mode auto --mode snapshot --storage cephfs --node tghci-node3
INFO: Starting Backup of VM 100102002 (qemu)
INFO: Backup started at 2025-08-26 22:04:43
INFO: status = running
INFO: VM Name: yunguantest826-102-20250825184443
INFO: include disk 'scsi0' 'hdd:vm-100102002-disk-1' 100G
INFO: include disk 'efidisk0' 'hdd:vm-100102002-disk-0' 64M
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating vzdump archive '/mnt/pve/cephfs/dump/vzdump-qemu-100102002-2025_08_26-22_04_43.vma.zst'
INFO: issuing guest-agent 'fs-freeze' command
INFO: issuing guest-agent 'fs-thaw' command
INFO: started backup task 'f6bbc3c7-38ff-407a-a5b5-0f7e4bb8a14b'
INFO: resuming VM again
INFO: 7% (7.1 GiB of 100.1 GiB) in 3s, read: 2.4 GiB/s, write: 152.7 MiB/s
INFO: 8% (8.1 GiB of 100.1 GiB) in 8s, read: 198.1 MiB/s, write: 84.0 KiB/s
INFO: 9% (9.1 GiB of 100.1 GiB) in 13s, read: 211.6 MiB/s, write: 20.8 KiB/s
INFO: 10% (10.1 GiB of 100.1 GiB) in 20s, read: 140.0 MiB/s, write: 64.9 MiB/s
INFO: 11% (11.2 GiB of 100.1 GiB) in 30s, read: 110.2 MiB/s, write: 88.1 MiB/s
INFO: 12% (12.1 GiB of 100.1 GiB) in 38s, read: 116.0 MiB/s, write: 92.8 MiB/s
INFO: 13% (13.1 GiB of 100.1 GiB) in 47s, read: 114.4 MiB/s, write: 97.1 MiB/s
INFO: 14% (14.0 GiB of 100.1 GiB) in 57s, read: 100.2 MiB/s, write: 82.5 MiB/s
INFO: 15% (15.0 GiB of 100.1 GiB) in 1m 9s, read: 84.7 MiB/s, write: 82.4 MiB/s
INFO: 16% (16.0 GiB of 100.1 GiB) in 1m 21s, read: 83.4 MiB/s, write: 71.5 MiB/s
INFO: 17% (17.1 GiB of 100.1 GiB) in 1m 34s, read: 83.8 MiB/s, write: 75.9 MiB/s
INFO: 18% (18.1 GiB of 100.1 GiB) in 1m 44s, read: 104.6 MiB/s, write: 85.0 MiB/s
INFO: 19% (19.0 GiB of 100.1 GiB) in 1m 52s, read: 120.5 MiB/s, write: 88.0 MiB/s
INFO: 20% (20.1 GiB of 100.1 GiB) in 2m 1s, read: 118.9 MiB/s, write: 74.5 MiB/s
INFO: 21% (21.0 GiB of 100.1 GiB) in 2m 12s, read: 88.2 MiB/s, write: 66.2 MiB/s
INFO: 22% (22.0 GiB of 100.1 GiB) in 2m 22s, read: 101.8 MiB/s, write: 71.1 MiB/s
INFO: 23% (23.1 GiB of 100.1 GiB) in 2m 33s, read: 95.2 MiB/s, write: 68.1 MiB/s
INFO: 24% (24.0 GiB of 100.1 GiB) in 2m 41s, read: 124.7 MiB/s, write: 90.4 MiB/s
INFO: 25% (25.0 GiB of 100.1 GiB) in 2m 51s, read: 104.4 MiB/s, write: 99.3 MiB/s
INFO: 26% (26.0 GiB of 100.1 GiB) in 3m 1s, read: 101.5 MiB/s, write: 83.7 MiB/s
INFO: 27% (27.1 GiB of 100.1 GiB) in 3m 12s, read: 100.5 MiB/s, write: 81.2 MiB/s
INFO: 28% (28.0 GiB of 100.1 GiB) in 3m 23s, read: 86.2 MiB/s, write: 63.9 MiB/s
INFO: 29% (29.1 GiB of 100.1 GiB) in 3m 33s, read: 105.6 MiB/s, write: 66.3 MiB/s
INFO: 30% (30.1 GiB of 100.1 GiB) in 3m 41s, read: 130.5 MiB/s, write: 65.5 MiB/s
INFO: 31% (31.2 GiB of 100.1 GiB) in 3m 50s, read: 121.6 MiB/s, write: 70.2 MiB/s
INFO: 32% (32.1 GiB of 100.1 GiB) in 3m 57s, read: 131.3 MiB/s, write: 62.4 MiB/s
INFO: 33% (33.1 GiB of 100.1 GiB) in 4m 6s, read: 116.2 MiB/s, write: 74.8 MiB/s
INFO: 34% (34.1 GiB of 100.1 GiB) in 4m 14s, read: 131.9 MiB/s, write: 65.8 MiB/s
INFO: 35% (35.1 GiB of 100.1 GiB) in 4m 24s, read: 98.6 MiB/s, write: 69.5 MiB/s
INFO: 36% (36.1 GiB of 100.1 GiB) in 4m 33s, read: 111.2 MiB/s, write: 75.6 MiB/s
INFO: 37% (37.1 GiB of 100.1 GiB) in 4m 40s, read: 153.3 MiB/s, write: 69.8 MiB/s
INFO: 38% (38.1 GiB of 100.1 GiB) in 4m 47s, read: 141.4 MiB/s, write: 68.8 MiB/s
INFO: 39% (39.1 GiB of 100.1 GiB) in 4m 55s, read: 127.3 MiB/s, write: 43.9 MiB/s
INFO: 40% (40.1 GiB of 100.1 GiB) in 5m 3s, read: 128.1 MiB/s, write: 96.5 MiB/s
INFO: 41% (41.1 GiB of 100.1 GiB) in 5m 11s, read: 130.6 MiB/s, write: 118.2 MiB/s
INFO: 42% (42.1 GiB of 100.1 GiB) in 5m 20s, read: 114.5 MiB/s, write: 76.9 MiB/s
INFO: 43% (43.1 GiB of 100.1 GiB) in 5m 27s, read: 143.6 MiB/s, write: 156.6 KiB/s
INFO: 44% (44.1 GiB of 100.1 GiB) in 5m 36s, read: 117.4 MiB/s, write: 90.2 MiB/s
INFO: 45% (45.0 GiB of 100.1 GiB) in 5m 43s, read: 137.6 MiB/s, write: 137.5 MiB/s
INFO: 46% (46.1 GiB of 100.1 GiB) in 5m 51s, read: 134.7 MiB/s, write: 117.3 MiB/s
INFO: 47% (47.1 GiB of 100.1 GiB) in 5m 59s, read: 130.7 MiB/s, write: 129.6 MiB/s
INFO: 48% (48.1 GiB of 100.1 GiB) in 6m 6s, read: 148.0 MiB/s, write: 16.3 MiB/s
INFO: 49% (49.1 GiB of 100.1 GiB) in 6m 12s, read: 164.3 MiB/s, write: 0 B/s
INFO: 50% (50.0 GiB of 100.1 GiB) in 6m 20s, read: 121.5 MiB/s, write: 15.0 KiB/s
INFO: 51% (51.1 GiB of 100.1 GiB) in 6m 27s, read: 161.9 MiB/s, write: 13.0 MiB/s
INFO: 52% (52.1 GiB of 100.1 GiB) in 6m 35s, read: 120.3 MiB/s, write: 24.4 MiB/s
INFO: 53% (53.1 GiB of 100.1 GiB) in 6m 42s, read: 151.1 MiB/s, write: 0 B/s
INFO: 54% (54.1 GiB of 100.1 GiB) in 6m 50s, read: 121.6 MiB/s, write: 2.0 KiB/s
INFO: 55% (55.0 GiB of 100.1 GiB) in 6m 55s, read: 199.6 MiB/s, write: 7.2 KiB/s
INFO: 56% (56.1 GiB of 100.1 GiB) in 7m 1s, read: 184.7 MiB/s, write: 13.3 KiB/s
INFO: 57% (57.1 GiB of 100.1 GiB) in 7m 6s, read: 199.0 MiB/s, write: 0 B/s
INFO: 58% (58.1 GiB of 100.1 GiB) in 7m 14s, read: 134.4 MiB/s, write: 106.3 MiB/s
INFO: 59% (59.0 GiB of 100.1 GiB) in 7m 30s, read: 57.1 MiB/s, write: 54.0 MiB/s
INFO: 60% (60.0 GiB of 100.1 GiB) in 7m 38s, read: 128.5 MiB/s, write: 112.9 MiB/s
INFO: 61% (61.1 GiB of 100.1 GiB) in 7m 48s, read: 103.6 MiB/s, write: 102.3 MiB/s
INFO: 62% (62.1 GiB of 100.1 GiB) in 7m 58s, read: 108.2 MiB/s, write: 99.3 MiB/s
INFO: 63% (63.2 GiB of 100.1 GiB) in 8m 8s, read: 108.3 MiB/s, write: 97.1 MiB/s
INFO: 64% (64.1 GiB of 100.1 GiB) in 8m 15s, read: 140.3 MiB/s, write: 36.8 MiB/s
INFO: 65% (65.1 GiB of 100.1 GiB) in 8m 22s, read: 137.3 MiB/s, write: 0 B/s
INFO: 66% (66.0 GiB of 100.1 GiB) in 8m 28s, read: 166.3 MiB/s, write: 2.7 KiB/s
INFO: 67% (67.2 GiB of 100.1 GiB) in 8m 35s, read: 164.0 MiB/s, write: 10.9 KiB/s
INFO: 68% (68.1 GiB of 100.1 GiB) in 8m 40s, read: 182.2 MiB/s, write: 819.0 B/s
INFO: 69% (69.0 GiB of 100.1 GiB) in 8m 45s, read: 203.2 MiB/s, write: 0 B/s
INFO: 70% (70.1 GiB of 100.1 GiB) in 8m 51s, read: 179.1 MiB/s, write: 42.3 MiB/s
INFO: 71% (71.1 GiB of 100.1 GiB) in 8m 56s, read: 211.4 MiB/s, write: 0 B/s
INFO: 72% (72.1 GiB of 100.1 GiB) in 9m, read: 236.5 MiB/s, write: 512.0 KiB/s
INFO: 73% (73.2 GiB of 100.1 GiB) in 9m 5s, read: 225.6 MiB/s, write: 0 B/s
INFO: 74% (74.1 GiB of 100.1 GiB) in 9m 10s, read: 190.2 MiB/s, write: 3.2 KiB/s
INFO: 75% (75.3 GiB of 100.1 GiB) in 9m 15s, read: 241.2 MiB/s, write: 819.0 B/s
INFO: 76% (76.1 GiB of 100.1 GiB) in 9m 19s, read: 214.5 MiB/s, write: 0 B/s
INFO: 77% (77.2 GiB of 100.1 GiB) in 9m 24s, read: 218.8 MiB/s, write: 2.4 KiB/s
INFO: 78% (78.2 GiB of 100.1 GiB) in 9m 29s, read: 219.2 MiB/s, write: 819.0 B/s
INFO: 79% (79.1 GiB of 100.1 GiB) in 9m 33s, read: 215.0 MiB/s, write: 0 B/s
INFO: 80% (80.1 GiB of 100.1 GiB) in 9m 38s, read: 212.2 MiB/s, write: 819.0 B/s
INFO: 81% (81.2 GiB of 100.1 GiB) in 9m 43s, read: 222.0 MiB/s, write: 819.0 B/s
INFO: 82% (82.1 GiB of 100.1 GiB) in 9m 47s, read: 235.2 MiB/s, write: 9.0 KiB/s
INFO: 83% (83.2 GiB of 100.1 GiB) in 9m 52s, read: 221.8 MiB/s, write: 3.2 KiB/s
INFO: 84% (84.1 GiB of 100.1 GiB) in 9m 56s, read: 221.8 MiB/s, write: 0 B/s
INFO: 85% (85.2 GiB of 100.1 GiB) in 10m 1s, read: 233.6 MiB/s, write: 16.0 KiB/s
INFO: 86% (86.2 GiB of 100.1 GiB) in 10m 5s, read: 255.3 MiB/s, write: 16.0 KiB/s
INFO: 87% (87.1 GiB of 100.1 GiB) in 10m 9s, read: 235.8 MiB/s, write: 0 B/s
INFO: 88% (88.1 GiB of 100.1 GiB) in 10m 13s, read: 241.8 MiB/s, write: 2.2 MiB/s
INFO: 89% (89.2 GiB of 100.1 GiB) in 10m 18s, read: 228.2 MiB/s, write: 819.0 B/s
INFO: 90% (90.3 GiB of 100.1 GiB) in 10m 24s, read: 188.8 MiB/s, write: 0 B/s
INFO: 91% (91.2 GiB of 100.1 GiB) in 10m 28s, read: 238.5 MiB/s, write: 4.0 KiB/s
INFO: 92% (92.2 GiB of 100.1 GiB) in 10m 32s, read: 257.0 MiB/s, write: 0 B/s
INFO: 93% (93.1 GiB of 100.1 GiB) in 10m 36s, read: 228.5 MiB/s, write: 0 B/s
INFO: 94% (94.3 GiB of 100.1 GiB) in 10m 41s, read: 236.5 MiB/s, write: 819.0 B/s
INFO: 95% (95.1 GiB of 100.1 GiB) in 10m 45s, read: 217.3 MiB/s, write: 0 B/s
INFO: 96% (96.2 GiB of 100.1 GiB) in 10m 50s, read: 211.6 MiB/s, write: 819.0 B/s
INFO: 97% (97.3 GiB of 100.1 GiB) in 10m 55s, read: 224.9 MiB/s, write: 329.6 KiB/s
INFO: 98% (98.1 GiB of 100.1 GiB) in 11m 1s, read: 139.7 MiB/s, write: 91.8 MiB/s
INFO: 99% (99.2 GiB of 100.1 GiB) in 11m 10s, read: 126.2 MiB/s, write: 124.1 MiB/s
ERROR: VM 100102002 qmp command 'query-backup' failed - got timeout
INFO: aborting backup job
ERROR: VM 100102002 qmp command 'backup-cancel' failed - unable to connect to VM 100102002 qmp socket - timeout after 5983 retries
INFO: resuming VM again
ERROR: Backup of VM 100102002 failed - VM 100102002 qmp command 'cont' failed - unable to connect to VM 100102002 qmp socket - timeout after 450 retries
INFO: Failed at 2025-08-26 22:37:18
INFO: Backup job finished with errors
INFO: notified via target `mail-to-root`
TASK ERROR: job errors
---------------------------------------------
vm config:
agent: 1
bios: ovmf
boot: order=scsi0;net0
cores: 8
efidisk0: hdd:vm-100102002-disk-0,efitype=4m,size=64M
ipconfig0: ip=dhcp
lock: backup
memory: 16384
meta: creation-qemu=8.1.2,ctime=1728963331
name: yunguantest826-102-20250825184443
net0: virtio=BC:24:11:8C:69:25,bridge=vmbr0,firewall=1
numa: 0
onboot: 1
ostype: l26
scsi0: hdd:vm-100102002-disk-1,iothread=1,size=100G
scsi2: hdd:vm-100102002-cloudinit,media=cdrom
scsihw: virtio-scsi-single
smbios1: uuid=b3622183-5004-4731-b93d-075404c918b0
sockets: 1
-----------------------------------------------------
I have checked the relevant information and found that QEMU introduced the AIO context mechanism in version 8.2.0; could this mechanism be the cause of this issue?
If a virtual machine has running containers, the backup issue with this version is almost certain to occur, and it will appear after approximately 60 backups.
the backup logs:
INFO: starting new backup job: vzdump 100102002 --remove 0 --compress zstd --notes-template '{{guestname}}' --notification-mode auto --mode snapshot --storage cephfs --node tghci-node3
INFO: Starting Backup of VM 100102002 (qemu)
INFO: Backup started at 2025-08-26 22:04:43
INFO: status = running
INFO: VM Name: yunguantest826-102-20250825184443
INFO: include disk 'scsi0' 'hdd:vm-100102002-disk-1' 100G
INFO: include disk 'efidisk0' 'hdd:vm-100102002-disk-0' 64M
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating vzdump archive '/mnt/pve/cephfs/dump/vzdump-qemu-100102002-2025_08_26-22_04_43.vma.zst'
INFO: issuing guest-agent 'fs-freeze' command
INFO: issuing guest-agent 'fs-thaw' command
INFO: started backup task 'f6bbc3c7-38ff-407a-a5b5-0f7e4bb8a14b'
INFO: resuming VM again
INFO: 7% (7.1 GiB of 100.1 GiB) in 3s, read: 2.4 GiB/s, write: 152.7 MiB/s
INFO: 8% (8.1 GiB of 100.1 GiB) in 8s, read: 198.1 MiB/s, write: 84.0 KiB/s
INFO: 9% (9.1 GiB of 100.1 GiB) in 13s, read: 211.6 MiB/s, write: 20.8 KiB/s
INFO: 10% (10.1 GiB of 100.1 GiB) in 20s, read: 140.0 MiB/s, write: 64.9 MiB/s
INFO: 11% (11.2 GiB of 100.1 GiB) in 30s, read: 110.2 MiB/s, write: 88.1 MiB/s
INFO: 12% (12.1 GiB of 100.1 GiB) in 38s, read: 116.0 MiB/s, write: 92.8 MiB/s
INFO: 13% (13.1 GiB of 100.1 GiB) in 47s, read: 114.4 MiB/s, write: 97.1 MiB/s
INFO: 14% (14.0 GiB of 100.1 GiB) in 57s, read: 100.2 MiB/s, write: 82.5 MiB/s
INFO: 15% (15.0 GiB of 100.1 GiB) in 1m 9s, read: 84.7 MiB/s, write: 82.4 MiB/s
INFO: 16% (16.0 GiB of 100.1 GiB) in 1m 21s, read: 83.4 MiB/s, write: 71.5 MiB/s
INFO: 17% (17.1 GiB of 100.1 GiB) in 1m 34s, read: 83.8 MiB/s, write: 75.9 MiB/s
INFO: 18% (18.1 GiB of 100.1 GiB) in 1m 44s, read: 104.6 MiB/s, write: 85.0 MiB/s
INFO: 19% (19.0 GiB of 100.1 GiB) in 1m 52s, read: 120.5 MiB/s, write: 88.0 MiB/s
INFO: 20% (20.1 GiB of 100.1 GiB) in 2m 1s, read: 118.9 MiB/s, write: 74.5 MiB/s
INFO: 21% (21.0 GiB of 100.1 GiB) in 2m 12s, read: 88.2 MiB/s, write: 66.2 MiB/s
INFO: 22% (22.0 GiB of 100.1 GiB) in 2m 22s, read: 101.8 MiB/s, write: 71.1 MiB/s
INFO: 23% (23.1 GiB of 100.1 GiB) in 2m 33s, read: 95.2 MiB/s, write: 68.1 MiB/s
INFO: 24% (24.0 GiB of 100.1 GiB) in 2m 41s, read: 124.7 MiB/s, write: 90.4 MiB/s
INFO: 25% (25.0 GiB of 100.1 GiB) in 2m 51s, read: 104.4 MiB/s, write: 99.3 MiB/s
INFO: 26% (26.0 GiB of 100.1 GiB) in 3m 1s, read: 101.5 MiB/s, write: 83.7 MiB/s
INFO: 27% (27.1 GiB of 100.1 GiB) in 3m 12s, read: 100.5 MiB/s, write: 81.2 MiB/s
INFO: 28% (28.0 GiB of 100.1 GiB) in 3m 23s, read: 86.2 MiB/s, write: 63.9 MiB/s
INFO: 29% (29.1 GiB of 100.1 GiB) in 3m 33s, read: 105.6 MiB/s, write: 66.3 MiB/s
INFO: 30% (30.1 GiB of 100.1 GiB) in 3m 41s, read: 130.5 MiB/s, write: 65.5 MiB/s
INFO: 31% (31.2 GiB of 100.1 GiB) in 3m 50s, read: 121.6 MiB/s, write: 70.2 MiB/s
INFO: 32% (32.1 GiB of 100.1 GiB) in 3m 57s, read: 131.3 MiB/s, write: 62.4 MiB/s
INFO: 33% (33.1 GiB of 100.1 GiB) in 4m 6s, read: 116.2 MiB/s, write: 74.8 MiB/s
INFO: 34% (34.1 GiB of 100.1 GiB) in 4m 14s, read: 131.9 MiB/s, write: 65.8 MiB/s
INFO: 35% (35.1 GiB of 100.1 GiB) in 4m 24s, read: 98.6 MiB/s, write: 69.5 MiB/s
INFO: 36% (36.1 GiB of 100.1 GiB) in 4m 33s, read: 111.2 MiB/s, write: 75.6 MiB/s
INFO: 37% (37.1 GiB of 100.1 GiB) in 4m 40s, read: 153.3 MiB/s, write: 69.8 MiB/s
INFO: 38% (38.1 GiB of 100.1 GiB) in 4m 47s, read: 141.4 MiB/s, write: 68.8 MiB/s
INFO: 39% (39.1 GiB of 100.1 GiB) in 4m 55s, read: 127.3 MiB/s, write: 43.9 MiB/s
INFO: 40% (40.1 GiB of 100.1 GiB) in 5m 3s, read: 128.1 MiB/s, write: 96.5 MiB/s
INFO: 41% (41.1 GiB of 100.1 GiB) in 5m 11s, read: 130.6 MiB/s, write: 118.2 MiB/s
INFO: 42% (42.1 GiB of 100.1 GiB) in 5m 20s, read: 114.5 MiB/s, write: 76.9 MiB/s
INFO: 43% (43.1 GiB of 100.1 GiB) in 5m 27s, read: 143.6 MiB/s, write: 156.6 KiB/s
INFO: 44% (44.1 GiB of 100.1 GiB) in 5m 36s, read: 117.4 MiB/s, write: 90.2 MiB/s
INFO: 45% (45.0 GiB of 100.1 GiB) in 5m 43s, read: 137.6 MiB/s, write: 137.5 MiB/s
INFO: 46% (46.1 GiB of 100.1 GiB) in 5m 51s, read: 134.7 MiB/s, write: 117.3 MiB/s
INFO: 47% (47.1 GiB of 100.1 GiB) in 5m 59s, read: 130.7 MiB/s, write: 129.6 MiB/s
INFO: 48% (48.1 GiB of 100.1 GiB) in 6m 6s, read: 148.0 MiB/s, write: 16.3 MiB/s
INFO: 49% (49.1 GiB of 100.1 GiB) in 6m 12s, read: 164.3 MiB/s, write: 0 B/s
INFO: 50% (50.0 GiB of 100.1 GiB) in 6m 20s, read: 121.5 MiB/s, write: 15.0 KiB/s
INFO: 51% (51.1 GiB of 100.1 GiB) in 6m 27s, read: 161.9 MiB/s, write: 13.0 MiB/s
INFO: 52% (52.1 GiB of 100.1 GiB) in 6m 35s, read: 120.3 MiB/s, write: 24.4 MiB/s
INFO: 53% (53.1 GiB of 100.1 GiB) in 6m 42s, read: 151.1 MiB/s, write: 0 B/s
INFO: 54% (54.1 GiB of 100.1 GiB) in 6m 50s, read: 121.6 MiB/s, write: 2.0 KiB/s
INFO: 55% (55.0 GiB of 100.1 GiB) in 6m 55s, read: 199.6 MiB/s, write: 7.2 KiB/s
INFO: 56% (56.1 GiB of 100.1 GiB) in 7m 1s, read: 184.7 MiB/s, write: 13.3 KiB/s
INFO: 57% (57.1 GiB of 100.1 GiB) in 7m 6s, read: 199.0 MiB/s, write: 0 B/s
INFO: 58% (58.1 GiB of 100.1 GiB) in 7m 14s, read: 134.4 MiB/s, write: 106.3 MiB/s
INFO: 59% (59.0 GiB of 100.1 GiB) in 7m 30s, read: 57.1 MiB/s, write: 54.0 MiB/s
INFO: 60% (60.0 GiB of 100.1 GiB) in 7m 38s, read: 128.5 MiB/s, write: 112.9 MiB/s
INFO: 61% (61.1 GiB of 100.1 GiB) in 7m 48s, read: 103.6 MiB/s, write: 102.3 MiB/s
INFO: 62% (62.1 GiB of 100.1 GiB) in 7m 58s, read: 108.2 MiB/s, write: 99.3 MiB/s
INFO: 63% (63.2 GiB of 100.1 GiB) in 8m 8s, read: 108.3 MiB/s, write: 97.1 MiB/s
INFO: 64% (64.1 GiB of 100.1 GiB) in 8m 15s, read: 140.3 MiB/s, write: 36.8 MiB/s
INFO: 65% (65.1 GiB of 100.1 GiB) in 8m 22s, read: 137.3 MiB/s, write: 0 B/s
INFO: 66% (66.0 GiB of 100.1 GiB) in 8m 28s, read: 166.3 MiB/s, write: 2.7 KiB/s
INFO: 67% (67.2 GiB of 100.1 GiB) in 8m 35s, read: 164.0 MiB/s, write: 10.9 KiB/s
INFO: 68% (68.1 GiB of 100.1 GiB) in 8m 40s, read: 182.2 MiB/s, write: 819.0 B/s
INFO: 69% (69.0 GiB of 100.1 GiB) in 8m 45s, read: 203.2 MiB/s, write: 0 B/s
INFO: 70% (70.1 GiB of 100.1 GiB) in 8m 51s, read: 179.1 MiB/s, write: 42.3 MiB/s
INFO: 71% (71.1 GiB of 100.1 GiB) in 8m 56s, read: 211.4 MiB/s, write: 0 B/s
INFO: 72% (72.1 GiB of 100.1 GiB) in 9m, read: 236.5 MiB/s, write: 512.0 KiB/s
INFO: 73% (73.2 GiB of 100.1 GiB) in 9m 5s, read: 225.6 MiB/s, write: 0 B/s
INFO: 74% (74.1 GiB of 100.1 GiB) in 9m 10s, read: 190.2 MiB/s, write: 3.2 KiB/s
INFO: 75% (75.3 GiB of 100.1 GiB) in 9m 15s, read: 241.2 MiB/s, write: 819.0 B/s
INFO: 76% (76.1 GiB of 100.1 GiB) in 9m 19s, read: 214.5 MiB/s, write: 0 B/s
INFO: 77% (77.2 GiB of 100.1 GiB) in 9m 24s, read: 218.8 MiB/s, write: 2.4 KiB/s
INFO: 78% (78.2 GiB of 100.1 GiB) in 9m 29s, read: 219.2 MiB/s, write: 819.0 B/s
INFO: 79% (79.1 GiB of 100.1 GiB) in 9m 33s, read: 215.0 MiB/s, write: 0 B/s
INFO: 80% (80.1 GiB of 100.1 GiB) in 9m 38s, read: 212.2 MiB/s, write: 819.0 B/s
INFO: 81% (81.2 GiB of 100.1 GiB) in 9m 43s, read: 222.0 MiB/s, write: 819.0 B/s
INFO: 82% (82.1 GiB of 100.1 GiB) in 9m 47s, read: 235.2 MiB/s, write: 9.0 KiB/s
INFO: 83% (83.2 GiB of 100.1 GiB) in 9m 52s, read: 221.8 MiB/s, write: 3.2 KiB/s
INFO: 84% (84.1 GiB of 100.1 GiB) in 9m 56s, read: 221.8 MiB/s, write: 0 B/s
INFO: 85% (85.2 GiB of 100.1 GiB) in 10m 1s, read: 233.6 MiB/s, write: 16.0 KiB/s
INFO: 86% (86.2 GiB of 100.1 GiB) in 10m 5s, read: 255.3 MiB/s, write: 16.0 KiB/s
INFO: 87% (87.1 GiB of 100.1 GiB) in 10m 9s, read: 235.8 MiB/s, write: 0 B/s
INFO: 88% (88.1 GiB of 100.1 GiB) in 10m 13s, read: 241.8 MiB/s, write: 2.2 MiB/s
INFO: 89% (89.2 GiB of 100.1 GiB) in 10m 18s, read: 228.2 MiB/s, write: 819.0 B/s
INFO: 90% (90.3 GiB of 100.1 GiB) in 10m 24s, read: 188.8 MiB/s, write: 0 B/s
INFO: 91% (91.2 GiB of 100.1 GiB) in 10m 28s, read: 238.5 MiB/s, write: 4.0 KiB/s
INFO: 92% (92.2 GiB of 100.1 GiB) in 10m 32s, read: 257.0 MiB/s, write: 0 B/s
INFO: 93% (93.1 GiB of 100.1 GiB) in 10m 36s, read: 228.5 MiB/s, write: 0 B/s
INFO: 94% (94.3 GiB of 100.1 GiB) in 10m 41s, read: 236.5 MiB/s, write: 819.0 B/s
INFO: 95% (95.1 GiB of 100.1 GiB) in 10m 45s, read: 217.3 MiB/s, write: 0 B/s
INFO: 96% (96.2 GiB of 100.1 GiB) in 10m 50s, read: 211.6 MiB/s, write: 819.0 B/s
INFO: 97% (97.3 GiB of 100.1 GiB) in 10m 55s, read: 224.9 MiB/s, write: 329.6 KiB/s
INFO: 98% (98.1 GiB of 100.1 GiB) in 11m 1s, read: 139.7 MiB/s, write: 91.8 MiB/s
INFO: 99% (99.2 GiB of 100.1 GiB) in 11m 10s, read: 126.2 MiB/s, write: 124.1 MiB/s
ERROR: VM 100102002 qmp command 'query-backup' failed - got timeout
INFO: aborting backup job
ERROR: VM 100102002 qmp command 'backup-cancel' failed - unable to connect to VM 100102002 qmp socket - timeout after 5983 retries
INFO: resuming VM again
ERROR: Backup of VM 100102002 failed - VM 100102002 qmp command 'cont' failed - unable to connect to VM 100102002 qmp socket - timeout after 450 retries
INFO: Failed at 2025-08-26 22:37:18
INFO: Backup job finished with errors
INFO: notified via target `mail-to-root`
TASK ERROR: job errors
---------------------------------------------
vm config:
agent: 1
bios: ovmf
boot: order=scsi0;net0
cores: 8
efidisk0: hdd:vm-100102002-disk-0,efitype=4m,size=64M
ipconfig0: ip=dhcp
lock: backup
memory: 16384
meta: creation-qemu=8.1.2,ctime=1728963331
name: yunguantest826-102-20250825184443
net0: virtio=BC:24:11:8C:69:25,bridge=vmbr0,firewall=1
numa: 0
onboot: 1
ostype: l26
scsi0: hdd:vm-100102002-disk-1,iothread=1,size=100G
scsi2: hdd:vm-100102002-cloudinit,media=cdrom
scsihw: virtio-scsi-single
smbios1: uuid=b3622183-5004-4731-b93d-075404c918b0
sockets: 1
-----------------------------------------------------
I have checked the relevant information and found that QEMU introduced the AIO context mechanism in version 8.2.0; could this mechanism be the cause of this issue?
If a virtual machine has running containers, the backup issue with this version is almost certain to occur, and it will appear after approximately 60 backups.
Last edited: