error VM qmp command failed - VM qmp command 'query-status' failed - unable to connect to VM 5021106 qmp socket - timeout after 51 retries

ssevasta

New Member
Jul 25, 2024
22
1
3
Hi, We are using veeam integration to backup proxmox, and we are having this error randomly on different VMs: "VM ID qmp command failed - VM ID qmp command 'query-status' failed - unable to connect to VM ID qmp socket - timeout after 51 retries ". Veeam support think it is a proxmox issue and asked me to open this case. Can you please help?

Regards,
 
Hi @ssevasta ,

This is a free community support forum, so posting here does not open an official support case. For formal case tracking, you would need to have an active support subscription.

That said, a staff member may still notice your post and take a look. However, you haven’t provided any useful technical details, such as an environment description, logs, or debug information.

In general, the error you’re seeing is typically caused by a stuck QEMU process. There are many possible ways to end up in this situation, the error itself is just a symptom.

If you can start from scratch and reliably reproduce the problem with a defined set of steps, please share that here as well. The best way for developers to address the issue is with a reproducible scenario.

Alternatively, you can report the problem via Proxmox Bugzilla, though given the limited details and the involvement of a third-party, unsupported application, it may be difficult to gain much traction there.


Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
Last edited:
  • Like
Reactions: UdoB
pve-manager/8.4.14/b502d23c55afcba1 (running kernel: 6.8.12-15-pve)
veeam is 12.3 (latest patch)
we have 10 pve host all running on the same cluster. 2 jobs when enable this option on veeam:
"LockVmOnBackup":
"CloseOtherBackups":
I have the backup failures.
I ran this command for several times on pve host: pvesh set /nodes/host/qemu/5021106/config --lock "backup" and i have no failures.
There is not a pattern, different VMs everytime on different host.
See attached the veeam logs.
 

Attachments

https://lore.proxmox.com/pve-devel/mailman.50.1727091601.332.pve-devel@lists.proxmox.com/t/

"


* During backup, there is often a longer running connection open to our QMP socket of running VMs
(/var/run/qemu-server/XXXX.qmp, where XXXX is the vmid). This blocks our management stack from
doing certain tasks, like start/stop (probably does not matter during backup) but also
things like the VNC console, etc.

a better way would be to close the connections as soon as possible instead of keeping them
open. (Alternatively using our API/CLI could also be done, but I don't know what
exact QMP commands you're running)

if you absolutely need a longer running socket, please open a bug report on
https://bugzilla.proxmox.com so we can discuss and track that there, how we could make
a socket available that is not used by our stack


"


Ask to veeam to fix their shit !