Proxmox (Unable to connect to VNC)

[dedivps]

Hello guys,

I am facing problem with Proxmox. Randomly VMs will hang and Windows RDP will not be connectable to check what's going on within Windows.

After "kill -9 id-of-vm-process" on terminal and restarting it, thing will get back to normal for a while before the next event. This does not happen to all VMs on once, but to random ones.

This is the error that I get on log from one of them that's not working:

VM 5133 qmp command 'set_password' failed - unable to connect to VM 5133 qmp socket - timeout after 31 retries
TASK ERROR: Failed to run vncproxy.


root@dedipath:~# pveversion
pve-manager/7.1-7/df5740ad (running kernel: 5.13.19-2-pve)


root@dedipath:~# pveversion --verbose
proxmox-ve: 7.1-1 (running kernel: 5.13.19-2-pve)
pve-manager: 7.1-7 (running version: 7.1-7/df5740ad)
pve-kernel-helper: 7.1-6
pve-kernel-5.13: 7.1-5
pve-kernel-5.13.19-2-pve: 5.13.19-4
ceph-fuse: 15.2.15-pve1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve2
libproxmox-acme-perl: 1.4.0
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.1-5
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.0-14
libpve-guest-common-perl: 4.0-3
libpve-http-server-perl: 4.0-4
libpve-storage-perl: 7.0-15
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.11-1
lxcfs: 4.0.11-pve1
novnc-pve: 1.2.0-3
proxmox-backup-client: 2.1.2-1
proxmox-backup-file-restore: 2.1.2-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.4-4
pve-cluster: 7.1-2
pve-container: 4.1-2
pve-docs: 7.1-2
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: 3.3-3
pve-ha-manager: 3.3-1
pve-i18n: 2.6-2
pve-qemu-kvm: 6.1.0-3
pve-xtermjs: 4.12.0-1
qemu-server: 7.1-4
smartmontools: 7.2-1
spiceterm: 3.2-2
swtpm: 0.7.0~rc1+2
vncterm: 1.7-1
zfsutils-linux: 2.1.1-pve3


Any suggestions what could I check?

Thanks in advance for your time.

 

[oguz]

hi,

This is the error that I get on log from one of them that's not working:

VM 5133 qmp command 'set_password' failed - unable to connect to VM 5133 qmp socket - timeout after 31 retries
TASK ERROR: Failed to run vncproxy.

Any suggestions what could I check?

* do you see anything in your journalctl / syslog file when this issue occurs?

* if you upgrade packages to the latest versions do you still have the issue?

 

[dedivps]

I am sorry for delayed reply.

This is output from syslog:

May 24 11:47:43 dedipath pvestatd[1459]: VM 5225 qmp command failed - VM 5225 qmp command 'query-proxmox-support' failed - unable to connect to VM 5225 qmp socket - timeout after 31 retries

May 24 11:48:30 dedipath pvedaemon[2620142]: VM 5225 qmp command failed - VM 5225 qmp command 'query-pci' failed - unable to connect to VM 5225 qmp socket - timeout after 31 retries

May 24 11:49:53 dedipath pvedaemon[2627118]: VM 5225 qmp command failed - VM 5225 qmp command 'system_powerdown' failed - unable to connect to VM 5225 qmp socket - timeout after 31 retries

May 24 11:51:52 dedipath qm[2627838]: VM 5225 qmp command failed - VM 5225 qmp command 'set_password' failed - unable to connect to VM 5225 qmp socket - timeout after 31 retries
May 24 11:51:52 dedipath pvedaemon[2627821]: Failed to run vncproxy.
May 24 11:51:52 dedipath pvedaemon[2620142]: <root@pam> end task UPID:dedipath:002818ED:0949F071:628D0D23:vncproxy:5225:root@pam: Failed to run vncproxy.


etc.


Upgrading is unfortunately not an option as this is production node and I risk to break a lot of VPS if it goes wrong.