[SOLVED] qemu guest agent vss provider stopped (after backup?)

T

trigg3r

Well-Known Member
Aug 3, 2019
55
7
48
24
In Proxmox 8.3 I created two VMs with Windows Server 2019 enabling "QEMU guest agent" in Options. In the VMs I installed QEMU guest agent (108.0.2) and set the related services to start automatically at boot.

Every morning in these two VMs I find the "QEMU Guest Agent VSS Provider" service stopped. If I restart it, the service starts without problems.

I think the problem is related to the Proxmox Snapshot Backup that I automatically perform every evening (successfully), but I don't know how to solve it.

Can anyone help me?

qemu_agent.png
 
Hi!

Have you also installed the related virtio-win drivers (e.g. the serial driver)?
 
  • Like
Reactions: trigg3r
Thaks for your answer @dakralex

Following Windows Guest Best Practices 2019:
  • installing Windows I install the driver for disk (vioscsi\2k19\amd64), network (NetKVM\2k19\amd64) and balloon (Balloon\2k19\amd64)
  • after installing Windows I installed Qemu Guest Agent (guest-agent\qemu-ga-x86_64.msi)
    • then I set the QEMU Guest Agent and QEMU Guest Agent VSS Provider services to start automatically on boot

Windows Device Manager does not show any "red cross" or "or yellow exclamation mark", so it seems that no other drivers are needed (right?)

device_manager.png
 
Last edited:
The problem was not solved even by uninstalling and reinstalling the VitIO drivers via the "virtio-win-guest-tools.exe" program located in the root of "vitio-win-0.1.266.iso".

Even though the "QEMU Guest Agent VSS Provider" service is stopped, backups seem to continue to run without errors: is this possible??


Any ideas on what I can try/check?
 
Hi!

trigg3r said:
Any ideas on what I can try/check?
Click to expand...
Could you check if there's anything logged in the PVE host's syslog or in the guest itself? Does changing the service from starting automatically to starting manually change the behavior?
 
  • Like
Reactions: trigg3r
Thanks again for your help @dakralex

File task-pve3-vzdump-2025-02-28T02_00_03Z.log are the Proxmox logs of the last backup task, which is performed at 3am.

I wrote a PowerShell script to extract logs containing the string "QEMU" from the "Application" and "System" Windows event registers. The attachment "DC1_qemu_log.txt" contains the DC1 server logs of the last 3 days (last two lines are me restarting the service and setting it to "Manual" as you suggested).

From these logs it seems that the service is stopped at the end of the backup:
Code: 
2025-02-28 03:00:01 AM 7036 Information Service Control Manager System The QEMU Guest Agent VSS Provider service has entered the running state.
2025-02-28 03:06:06 AM 7036 Information Service Control Manager System The QEMU Guest Agent VSS Provider service has entered the stopped state.

Note: in the Windows service "QEMU guest agent VSS" I also tried to set the restart on error (attached qemu_vss_service_first_failure.png), but this had no effect (because no error actually occurred ...?).
 

Attachments

Last edited:
Thanks for getting back with those logs!

I'll have to take a closer look at this next week, but I noticed that the backup is already finishing some minutes before that (except one of the clocks in out-of-sync):

INFO: Starting Backup of VM 100 (qemu)
INFO: Backup started at 2025-02-28 03:00:03
INFO: status = running
INFO: VM Name: DC1
INFO: include disk 'scsi0' 'VM-STORAGE:vm-100-disk-0' 100G
[ ... ]
INFO: Finished Backup of VM 100 (00:01:02)
INFO: Backup finished at 2025-02-28 03:01:05
[ ... ]
Click to expand...

but the service is only stopped at 03:06:06 AM:

2/28/2025 3:00:01 AM 1 Information qemu-ga Application guest-fsfreeze called
2/28/2025 3:00:01 AM 7036 Information Service Control Manager System The QEMU Guest Agent VSS Provider service entered the esecuzione state.
2/28/2025 3:06:06 AM 7036 Information Service Control Manager System The QEMU Guest Agent VSS Provider service entered the arrestato state.
Click to expand...
 
  • Like
Reactions: trigg3r
Hi @dakralex

I went to fix the clock synchronization and now Proxmox/DC1/SRV1/SVR2 synchronize with the same NTP server.

Attached are QEMU logs of the three Windows Servers and Proxmox backup task logs, all related to a backup that I manually performed at 22:30 last night and the automatic backup at 3:00 this morning.

Thanks again for your help.
 

Attachments

trigg3r said:
I think the problem is related to the Proxmox Snapshot Backup that I automatically perform every evening (successfully), but I don't know how to solve it.
Click to expand...
Hi @trigg3r!
Btw do you experience any problems because the service is stopped? Because AFAIK this service is only run exactly for backups, so that the guest filesystem is synchronized and frozen (guest-fsfreeze-freeze) and then unfrozen again (guest-fsfreeze-thaw).

That's the only use case for this service from our side, does a workload of yours depend on this service too?
 
  • Like
Reactions: trigg3r
Hi @trigg3r!

Oh, yes, but it is only vital that the Qemu Guest Agent itself is running as a service. The Qemu Guest Agent VSS Provider is only the implementation for the two guest-fsfreeze-freeze and guest-fsfreeze-thaw QMP commands that I've mentioned above, as those are implemented by Microsoft Windows' Volume Shadow Copy (VSS) functionality and the "Qemu Guest Agent VSS Provider" service is therefore only needed when those are executed.
 
  • Like
Reactions: _gabriel and trigg3r
So if backup works, I can safely set the service QEMU guest agent VSS to start manually and I'm good to go, did I understand correctly?
 
Yes, correct! You can leave/set it in manual mode since the QEMU Guest Agent will start it when needed anyway.
 
  • Like
Reactions: trigg3r
You must log in or register to reply here.
Top Bottom