[SOLVED] Problem with pveproxy "detected empty handle" after update

[Jonas K]

Hello there,

I just updated to a new Proxmox Version. But once it was done I noticed connection interrupts across all containers and proxmox services.
After I have done some research I found: "pveproxy[]: detected empty handle".

I already took the following steps (without success)

• looking for new updates
• I tried restarting the following services:
  • pve-cluster
  • pveproxy
• I checked my system resources which seem to be fine

While trying to solve my problem. I stumbled across another error message: "ipcc_send_rec[1] failed: Permission denied"

I really hope you can help me. Below you can find an excerpt of system logs and details about my setup.

System log:

Aug 15 10:58:48 pve pveproxy[8692]: detected empty handle
Aug 15 10:58:48 pve pveproxy[8701]: detected empty handle
Aug 15 10:58:49 pve pvedaemon[12493]: connection timed out
Aug 15 10:58:49 pve pvedaemon[1229]: <root@pam> end task UPID:pve:000030CD:000325FA:66BDC33F:vncproxy:110:root@pam: connection timed out
Aug 15 10:58:49 pve pveproxy[8691]: detected empty handle
Aug 15 10:58:49 pve pveproxy[8692]: detected empty handle
Aug 15 10:58:49 pve pveproxy[8701]: detected empty handle

Setup Informationen

proxmox-ve: 8.2.0 (running kernel: 6.8.12-1-pve)
pve-manager: 8.2.4 (running version: 8.2.4/faa83925c9641325)
proxmox-kernel-helper: 8.1.0
proxmox-kernel-6.8: 6.8.12-1
proxmox-kernel-6.8.12-1-pve-signed: 6.8.12-1
proxmox-kernel-6.8.8-2-pve-signed: 6.8.8-2
proxmox-kernel-6.5.13-6-pve-signed: 6.5.13-6
proxmox-kernel-6.5: 6.5.13-6
proxmox-kernel-6.5.13-1-pve-signed: 6.5.13-1
proxmox-kernel-6.5.11-4-pve-signed: 6.5.11-4
ceph-fuse: 17.2.7-pve1
corosync: 3.1.7-pve3
criu: 3.17.1-2
glusterfs-client: 10.3-5
ifupdown2: 3.2.0-1+pmx9
ksm-control-daemon: 1.5-1
libjs-extjs: 7.0.0-4
libknet1: 1.28-pve1
libproxmox-acme-perl: 1.5.1
libproxmox-backup-qemu0: 1.4.1
libproxmox-rs-perl: 0.3.3
libpve-access-control: 8.1.4
libpve-apiclient-perl: 3.3.2
libpve-cluster-api-perl: 8.0.7
libpve-cluster-perl: 8.0.7
libpve-common-perl: 8.2.2
libpve-guest-common-perl: 5.1.4
libpve-http-server-perl: 5.1.0
libpve-network-perl: 0.9.8
libpve-rs-perl: 0.8.9
libpve-storage-perl: 8.2.3
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 6.0.0-1
lxcfs: 6.0.0-pve2
novnc-pve: 1.4.0-3
proxmox-backup-client: 3.2.7-1
proxmox-backup-file-restore: 3.2.7-1
proxmox-firewall: 0.5.0
proxmox-kernel-helper: 8.1.0
proxmox-mail-forward: 0.2.3
proxmox-mini-journalreader: 1.4.0
proxmox-offline-mirror-helper: 0.6.6
proxmox-widget-toolkit: 4.2.3
pve-cluster: 8.0.7
pve-container: 5.1.12
pve-docs: 8.2.3
pve-edk2-firmware: 4.2023.08-4
pve-esxi-import-tools: 0.7.1
pve-firewall: 5.0.7
pve-firmware: 3.13-1
pve-ha-manager: 4.0.5
pve-i18n: 3.2.2
pve-qemu-kvm: 9.0.2-2
pve-xtermjs: 5.3.0-3
qemu-server: 8.2.4
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.2.4-pve1

 

[Jonas K]

I got some updates. I downloaded "Proxmox VE 8.2 ISO Installer" and installed it on my machine -> same error
Then I tried to install "Proxmox VE 7.4 ISO Installer" (which should have been the version I used before the update) -> same error

Has anybody an idea, what I am missing here?

 

[cybersecmike]

Exactly same thing is happening to me. I have a cluster with 3 nodes, updated one of them and boom, lost GUI access with the same exact "detected empty handle" on pveproxy. Any ideas what help restore GUI access?

 

[Jonas K]

I have found the solution to my problem.
My network cable had a defect, so a simple replacement did the trick.

Regarding your problem, I have got some questions:

1. Have you checked that all networking devices (routers, switches, cables) are working
2. Have you tried to restart the pveproxy service?
3. Have you tried restarting every node in your cluster?

I hope this was helpful

 

[cybersecmike]

I have found the solution to my problem.
My network cable had a defect, so a simple replacement did the trick.

Regarding your problem, I have got some questions:

1. Have you checked that all networking devices (routers, switches, cables) are working
2. Have you tried to restart the pveproxy service?
3. Have you tried restarting every node in your cluster?

I hope this was helpful

1. Yes, I've checked the router, switches but not the cables (I reckon since I have access to the cluster over SSH, the cables would be fine. But I'll change the cables to be 100% sure that isn't the problem).
2. Yes, restarted pveproxy service multiple times. It starts normally, but starts producing the error once I try to connect via the brower.
3. Yes, restarted every node in the cluster multiple times as well.

 

[Jonas K]

1. Yes, I've checked the router, switches but not the cables (I reckon since I have access to the cluster over SSH, the cables would be fine. But I'll change the cables to be 100% sure that isn't the problem).
2. Yes, restarted pveproxy service multiple times. It starts normally, but starts producing the error once I try to connect via the brower.
3. Yes, restarted every node in the cluster multiple times as well.

Ok, that sounds troublesome.

Just in case you haven't already found it, there is another thread about this problem. Maybe you'll find an answer there (or at least there are more people in that thread, so you should get responses from more people).

If possible, just drag your cluster to another room and connect it as close to your main router as possible. That way we can rule out faulty cables and switches.

I hope that's enough to resolve your issue. I talked to a network administrator a few days ago and he says that the most plausible reason behind this message would be faulty network devices or cables