novnc no migration qmp socket

[Paolo Marinelli]

since upgrade to proxmox 6 i had some problem that randomly occurs on vm
i cannot run console from web interface, this happen only on some vm and only on a server of my 3 node cluster
servers are identical and with the same proxmox upgraded version
i tried to analyze and the problem is that qmp of that vm socket stopped to work
if i try to open "monitor" tab on vm i get message "

"Type 'help' for help.
# help
ERROR: VM 101 qmp command 'human-monitor-command' failed - got timeout
"
VM is a Linux based, if i reboot VM by Os command "reboot", so qemu process doesnt change
the problem persist, if i stop VM and after start (new qemu process) the problem goes away

I think that the problem arise after some time

Summary

3 node cluster, all update all same kernel and packages all same hardware
only on node 1 there is the problem
only on some VM on node 1 there is problem (12VM on total of 16)
the problem is that VM run without problem, but cannot launch console, cannot migrate
if reboot VM (so qemu pid doesnt change) the problem persist
if stop and start VM (so there is a new qemu pid) the problem solve


root@srvprox01:~/2019problemaConsole/prox01# pveversion -v
proxmox-ve: 6.0-2 (running kernel: 5.0.21-1-pve)
pve-manager: 6.0-6 (running version: 6.0-6/c71f879f)
pve-kernel-5.0: 6.0-7
pve-kernel-helper: 6.0-7
pve-kernel-4.15: 5.4-6
pve-kernel-5.0.21-1-pve: 5.0.21-2
pve-kernel-5.0.15-1-pve: 5.0.15-1
pve-kernel-4.15.18-18-pve: 4.15.18-44
ceph: 14.2.2-pve1
ceph-fuse: 14.2.2-pve1
corosync: 3.0.2-pve2
criu: 3.11-3
glusterfs-client: 5.5-3
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.11-pve1
libpve-access-control: 6.0-2
libpve-apiclient-perl: 3.0-2
libpve-common-perl: 6.0-4
libpve-guest-common-perl: 3.0-1
libpve-http-server-perl: 3.0-2
libpve-storage-perl: 6.0-7
libqb0: 1.0.5-1
lvm2: 2.03.02-pve3
lxc-pve: 3.1.0-64
lxcfs: 3.0.3-pve60
novnc-pve: 1.0.0-60
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.0-7
pve-cluster: 6.0-7
pve-container: 3.0-5
pve-docs: 6.0-4
pve-edk2-firmware: 2.20190614-1
pve-firewall: 4.0-7
pve-firmware: 3.0-2
pve-ha-manager: 3.0-2
pve-i18n: 2.0-2
pve-qemu-kvm: 4.0.0-5
pve-xtermjs: 3.13.2-1
qemu-server: 6.0-7
smartmontools: 7.0-pve2
spiceterm: 3.1-1
vncterm: 1.6-1
zfsutils-linux: 0.8.1-pve2

 

[aaron]

Sounds a lot like the same cause as in this thread: https://forum.proxmox.com/threads/certain-vms-from-a-cluster-cannot-be-backed-up-and-managed.57016/

tldr: we know what the problem is and are working on a fix but are not quite there yet.

 

[Paolo Marinelli]

Sounds a lot like the same cause as in this thread: https://forum.proxmox.com/threads/certain-vms-from-a-cluster-cannot-be-backed-up-and-managed.57016/

tldr: we know what the problem is and are working on a fix but are not quite there yet.

Hi Aaron, after your answer i tried backup and confirm that also backup stopped to work now i have to shutdown and restart all VM