VM not working properly after backup

H

homepve

Member
Nov 25, 2022
8
0
6
Hi,

I have several LXCs and one VM, the latter running HAOS.

I usually backup all of them every day using PBS, first backup local and second one cloud. I perform them like 2 hours apart.

Sometimes the VM doesn't work properly after the backup. If I go to the UI I see it running but I have a warning that QEMU GUEST AGENT is not running. The only way to make it work is performing a hard reset from the console.

Any help? Thank you.
 
Hi,

Thank you for the quick reply.

I don't have a picture of the warning, it's working now. On the HAOS console it was freezed at "Booting" plus something more but now is working since I did the hard reset.

Regarding to the backup log:


Code: 
INFO: Starting Backup of VM 109 (qemu)
INFO: Backup started at 2025-05-29 03:43:10
INFO: status = running
INFO: backup mode: stop
INFO: ionice priority: 7
INFO: VM Name: haos
INFO: include disk 'scsi0' 'local-lvm:vm-109-disk-1' 32G
INFO: include disk 'efidisk0' 'local-lvm:vm-109-disk-0' 4M
INFO: stopping virtual guest
INFO: creating Proxmox Backup Server archive 'vm/109/2025-05-29T01:43:10Z'
INFO: starting kvm to execute backup task
INFO: started backup task '37622c4f-891d-48f8-9e62-93ffb1e63ef4'
INFO: resuming VM again after 32 seconds
INFO: efidisk0: dirty-bitmap status: created new
INFO: scsi0: dirty-bitmap status: created new
INFO:   1% (524.5 MiB of 32.0 GiB) in 3s, read: 174.8 MiB/s, write: 1.5 MiB/s
INFO:   2% (872.5 MiB of 32.0 GiB) in 6s, read: 116.0 MiB/s, write: 21.3 MiB/s
INFO:   3% (1.2 GiB of 32.0 GiB) in 9s, read: 110.7 MiB/s, write: 4.0 MiB/s
INFO:   4% (1.4 GiB of 32.0 GiB) in 12s, read: 82.7 MiB/s, write: 30.7 MiB/s
INFO:   5% (1.7 GiB of 32.0 GiB) in 15s, read: 92.0 MiB/s, write: 13.3 MiB/s
INFO:   6% (2.0 GiB of 32.0 GiB) in 18s, read: 122.7 MiB/s, write: 12.0 MiB/s
INFO:   7% (2.3 GiB of 32.0 GiB) in 21s, read: 84.0 MiB/s, write: 2.7 MiB/s
INFO:   8% (2.6 GiB of 32.0 GiB) in 24s, read: 94.7 MiB/s, write: 21.3 MiB/s
INFO:   9% (2.9 GiB of 32.0 GiB) in 27s, read: 112.0 MiB/s, write: 42.7 MiB/s
INFO:  10% (3.2 GiB of 32.0 GiB) in 33s, read: 56.0 MiB/s, write: 34.0 MiB/s
INFO:  11% (3.8 GiB of 32.0 GiB) in 36s, read: 196.0 MiB/s, write: 8.0 MiB/s
INFO:  14% (4.7 GiB of 32.0 GiB) in 39s, read: 314.7 MiB/s, write: 8.0 MiB/s
INFO:  15% (5.0 GiB of 32.0 GiB) in 42s, read: 81.3 MiB/s, write: 33.3 MiB/s
INFO:  16% (5.2 GiB of 32.0 GiB) in 45s, read: 73.3 MiB/s, write: 28.0 MiB/s
INFO:  18% (5.9 GiB of 32.0 GiB) in 48s, read: 262.7 MiB/s, write: 10.7 MiB/s
INFO:  21% (6.9 GiB of 32.0 GiB) in 51s, read: 332.0 MiB/s, write: 9.3 MiB/s
INFO:  22% (7.2 GiB of 32.0 GiB) in 55s, read: 80.0 MiB/s, write: 10.0 MiB/s
INFO:  26% (8.3 GiB of 32.0 GiB) in 58s, read: 378.7 MiB/s, write: 12.0 MiB/s
INFO:  28% (9.0 GiB of 32.0 GiB) in 1m 1s, read: 218.7 MiB/s, write: 8.0 MiB/s
INFO:  29% (9.3 GiB of 32.0 GiB) in 1m 4s, read: 121.3 MiB/s, write: 8.0 MiB/s
INFO:  30% (9.6 GiB of 32.0 GiB) in 1m 7s, read: 97.3 MiB/s, write: 20.0 MiB/s
INFO:  31% (10.1 GiB of 32.0 GiB) in 1m 11s, read: 113.0 MiB/s, write: 12.0 MiB/s
INFO:  32% (10.4 GiB of 32.0 GiB) in 1m 14s, read: 117.3 MiB/s, write: 17.3 MiB/s
INFO:  33% (10.8 GiB of 32.0 GiB) in 1m 17s, read: 149.3 MiB/s, write: 9.3 MiB/s
INFO:  35% (11.3 GiB of 32.0 GiB) in 1m 21s, read: 125.0 MiB/s, write: 6.0 MiB/s
INFO:  37% (12.0 GiB of 32.0 GiB) in 1m 24s, read: 216.0 MiB/s, write: 5.3 MiB/s
INFO:  39% (12.6 GiB of 32.0 GiB) in 1m 27s, read: 225.3 MiB/s, write: 8.0 MiB/s
INFO:  41% (13.2 GiB of 32.0 GiB) in 1m 30s, read: 182.7 MiB/s, write: 1.3 MiB/s
INFO:  42% (13.6 GiB of 32.0 GiB) in 1m 34s, read: 105.0 MiB/s, write: 21.0 MiB/s
INFO:  45% (14.4 GiB of 32.0 GiB) in 1m 37s, read: 297.3 MiB/s, write: 8.0 MiB/s
INFO:  46% (14.8 GiB of 32.0 GiB) in 1m 40s, read: 132.0 MiB/s, write: 14.7 MiB/s
INFO:  47% (15.2 GiB of 32.0 GiB) in 1m 43s, read: 137.3 MiB/s, write: 13.3 MiB/s
INFO:  48% (15.6 GiB of 32.0 GiB) in 1m 46s, read: 126.7 MiB/s, write: 21.3 MiB/s
INFO:  49% (15.8 GiB of 32.0 GiB) in 1m 49s, read: 73.3 MiB/s, write: 9.3 MiB/s
INFO:  50% (16.1 GiB of 32.0 GiB) in 1m 52s, read: 92.0 MiB/s, write: 24.0 MiB/s
INFO:  51% (16.3 GiB of 32.0 GiB) in 1m 55s, read: 81.3 MiB/s, write: 32.0 MiB/s
INFO:  52% (16.7 GiB of 32.0 GiB) in 2m 1s, read: 58.0 MiB/s, write: 30.7 MiB/s
INFO:  54% (17.5 GiB of 32.0 GiB) in 2m 4s, read: 280.0 MiB/s, write: 1.3 MiB/s
INFO:  56% (18.1 GiB of 32.0 GiB) in 2m 7s, read: 204.0 MiB/s, write: 0 B/s
INFO:  57% (18.5 GiB of 32.0 GiB) in 2m 10s, read: 138.7 MiB/s, write: 0 B/s
INFO:  59% (19.1 GiB of 32.0 GiB) in 2m 13s, read: 198.7 MiB/s, write: 5.3 MiB/s
INFO:  61% (19.6 GiB of 32.0 GiB) in 2m 16s, read: 190.7 MiB/s, write: 0 B/s
INFO:  62% (20.1 GiB of 32.0 GiB) in 2m 19s, read: 173.3 MiB/s, write: 2.7 MiB/s
INFO:  64% (20.6 GiB of 32.0 GiB) in 2m 22s, read: 174.7 MiB/s, write: 13.3 MiB/s
INFO:  66% (21.2 GiB of 32.0 GiB) in 2m 25s, read: 190.7 MiB/s, write: 8.0 MiB/s
INFO:  70% (22.6 GiB of 32.0 GiB) in 2m 28s, read: 486.7 MiB/s, write: 0 B/s
INFO:  71% (22.9 GiB of 32.0 GiB) in 2m 31s, read: 104.0 MiB/s, write: 10.7 MiB/s
INFO:  72% (23.3 GiB of 32.0 GiB) in 2m 34s, read: 116.0 MiB/s, write: 2.7 MiB/s
INFO:  77% (24.7 GiB of 32.0 GiB) in 2m 37s, read: 482.7 MiB/s, write: 4.0 MiB/s
INFO:  78% (25.1 GiB of 32.0 GiB) in 2m 40s, read: 132.0 MiB/s, write: 9.3 MiB/s
INFO:  79% (25.3 GiB of 32.0 GiB) in 2m 46s, read: 42.7 MiB/s, write: 40.0 MiB/s
INFO:  81% (26.2 GiB of 32.0 GiB) in 2m 53s, read: 126.9 MiB/s, write: 30.3 MiB/s
INFO:  85% (27.4 GiB of 32.0 GiB) in 2m 56s, read: 413.3 MiB/s, write: 0 B/s
INFO:  91% (29.3 GiB of 32.0 GiB) in 3m, read: 495.0 MiB/s, write: 0 B/s
INFO:  94% (30.4 GiB of 32.0 GiB) in 3m 3s, read: 345.3 MiB/s, write: 0 B/s
INFO: 100% (32.0 GiB of 32.0 GiB) in 3m 6s, read: 562.7 MiB/s, write: 0 B/s
INFO: Waiting for server to finish backup validation...
INFO: backup is sparse: 17.03 GiB (53%) total zero data
INFO: backup was done incrementally, reused 29.48 GiB (92%)
INFO: transferred 32.00 GiB in 191 seconds (171.6 MiB/s)
INFO: adding notes to backup
INFO: prune older backups with retention: keep-last=6
INFO: running 'proxmox-backup-client prune' for 'vm/109'
INFO: pruned 1 backup(s) not covered by keep-retention policy
INFO: Finished Backup of VM 109 (00:03:46)
INFO: Backup finished at 2025-05-29 03:46:56

For the cli commands, attached as screenshots. Thank you for your help.
 

Attachments

  • Screenshot 2025-05-29 22.12.17.png
    Screenshot 2025-05-29 22.12.17.png
    21.7 KB · Views: 4
  • Screenshot 2025-05-29 22.11.42.png
    Screenshot 2025-05-29 22.11.42.png
    48.3 KB · Views: 4
I don't see any issue here. You use the correct image variant (ova), the guest agent service is enabled and running and the log looks okay too. If it happens again please share pictures of it. The qm config VMID --current command is missing. Please share that too. I can't verify the guest agent is enabled for the VM due to the missing qm command but assuming it's enabled I'd recommend the snapshot backup mode to avoid downtime and to benefit from dirty bitmaps (very fast backups).
 
Last edited:
Thank you again. The output of qm config VMID --current is attached.

Some more info, these are the backups I run:

02:00 - LXC (12 of them) - local - stop.
02:30 - VM (only HAOS) - local - stop.
03:30 - Both LXC and VM - remote PBS - stop.

The one that sometimes, not always, doesn't let VM start again is the one I run at 03:30. It freezes at booting and I can see qemu guest agent not running from the summary section of the UI.
 

Attachments

  • PHOTO-2025-05-30-08-44-52.jpg
    PHOTO-2025-05-30-08-44-52.jpg
    48.5 KB · Views: 2
Not sure why you use VirtIO-GPU and I'd recommend to set a CPU model such as host or x86-64-v2-AES. The rest of the config looks good.
Take a look at the Guest Agent logs inside HAOS via journalctl -ru qemu-guest.service and check for issues.
Can you share the past logs of the backup at 03:30? You can find them in node > Task History. Please use code blocks whenever possible.
 
Probably followed a guide when creating this VM years ago, that's why I use VirtIO-GPU. I don't need any GPU, right? For HAOS I just use the app or the browser, but my Proxmox server is running headless. I can change the CPU model too. Should I update QEMU?

I'll post the logs probably tonight or tomorrow, I cannot use a computer now and from the phone is not really convenient.

Thank you.
 
I'd just leave the GPU at the default setting. I'm not sure if VirtIO-GPU can cause such an issue. I just noticed it and it's not needed for HAOS.
QEMU would be updated with normal PVE updates. The QEMU Guest Agent inside HAOS will be updated with HAOS upgrades.
 
You must log in or register to reply here.
Top Bottom