[SOLVED] After update 5.1-43, pvedaemon and pvestatd reports rados_connect failed - Operation not supported

mbaldini

Active Member
Nov 7, 2015
170
21
38
Hi.

In my production/test cluster with Proxmox and CEPH (same nodes) I have a node that I use for testing updates before applying to other servers.
Repository pve-no-subscription.
After today updates I get in log (just an extract)
Code:
May 02 17:56:47 pve-hs-3 pvedaemon[4929]: rados_connect failed - Operation not supported
May 02 17:56:47 pve-hs-3 pvestatd[25780]: rados_connect failed - Operation not supported
May 02 17:56:47 pve-hs-3 pvestatd[25780]: rados_connect failed - Operation not supported
May 02 17:56:48 pve-hs-3 pvestatd[25780]: rados_connect failed - Operation not supported
May 02 17:56:53 pve-hs-3 pvestatd[25780]: got timeout
May 02 17:56:53 pve-hs-3 pvestatd[59076]: rados_connect failed - Operation not supported
May 02 17:56:53 pve-hs-3 pvestatd[25780]: rados_connect failed - Operation not supported
May 02 17:56:53 pve-hs-3 pvedaemon[4926]: rados_connect failed - Operation not supported
May 02 17:56:53 pve-hs-3 pvestatd[25780]: status update time (5.796 seconds)
May 02 17:56:55 pve-hs-3 pvedaemon[4926]: rados_connect failed - Operation not supported
May 02 17:56:55 pve-hs-3 pvedaemon[4926]: rados_connect failed - Operation not supported
May 02 17:56:55 pve-hs-3 pvedaemon[4926]: rados_connect failed - Operation not supported
May 02 17:56:55 pve-hs-3 pvedaemon[4926]: rados_connect failed - Operation not supported
May 02 17:56:57 pve-hs-3 pvestatd[25780]: rados_connect failed - Operation not supported
May 02 17:56:57 pve-hs-3 pvestatd[25780]: rados_connect failed - Operation not supported
May 02 17:56:57 pve-hs-3 pvestatd[25780]: rados_connect failed - Operation not supported
May 02 17:56:57 pve-hs-3 pvestatd[25780]: rados_connect failed - Operation not supported
and so forever

This node has versions:
Code:
proxmox-ve: 5.1-43 (running kernel: 4.15.15-1-pve)
pve-manager: 5.1-52 (running version: 5.1-52/ba597a64)
pve-kernel-4.13: 5.1-44
pve-kernel-4.15: 5.1-3
pve-kernel-4.15.15-1-pve: 4.15.15-6
pve-kernel-4.13.16-2-pve: 4.13.16-47
pve-kernel-4.13.16-1-pve: 4.13.16-46
pve-kernel-4.13.13-6-pve: 4.13.13-42
pve-kernel-4.13.13-5-pve: 4.13.13-38
pve-kernel-4.13.13-2-pve: 4.13.13-33
pve-kernel-4.13.13-1-pve: 4.13.13-31
pve-kernel-4.13.8-3-pve: 4.13.8-30
pve-kernel-4.13.8-2-pve: 4.13.8-28
pve-kernel-4.13.4-1-pve: 4.13.4-26
ceph: 12.2.4-pve1
corosync: 2.4.2-pve5
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.0-8
libpve-apiclient-perl: 2.0-4
libpve-common-perl: 5.0-30
libpve-guest-common-perl: 2.0-15
libpve-http-server-perl: 2.0-8
libpve-storage-perl: 5.0-19
libqb0: 1.0.1-1
lvm2: 2.02.168-pve6
lxc-pve: 3.0.0-2
lxcfs: 3.0.0-1
novnc-pve: 0.6-4
proxmox-widget-toolkit: 1.0-15
pve-cluster: 5.0-26
pve-container: 2.0-22
pve-docs: 5.1-17
pve-firewall: 3.0-8
pve-firmware: 2.0-4
pve-ha-manager: 2.0-5
pve-i18n: 1.0-4
pve-libspice-server1: 0.12.8-3
pve-qemu-kvm: 2.11.1-5
pve-xtermjs: 1.0-3
qemu-server: 5.0-25
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3
zfsutils-linux: 0.7.7-pve1~bpo9

In this node I'm running kernel 4.15 installed with the package pve-kernel-4.15

Because of the error that you can see in logs, in the GUI, this node has no information on ceph pools
si3uQTj.png



Before the updates, there were no error. What's happened?
 

tom

Proxmox Staff Member
Staff member
Aug 29, 2006
15,554
925
163
Please check your Ceph version, post the output of:

> ceph versions
 

mbaldini

Active Member
Nov 7, 2015
170
21
38
Please check your Ceph version, post the output of:

> ceph versions
It seems correct to me
Code:
$ ceph versions
{
    "mon": {
        "ceph version 12.2.4 (4832b6f0acade977670a37c20ff5dbe69e727416) luminous (stable)": 3
    },
    "mgr": {
        "ceph version 12.2.4 (4832b6f0acade977670a37c20ff5dbe69e727416) luminous (stable)": 3
    },
    "osd": {
        "ceph version 12.2.4 (4832b6f0acade977670a37c20ff5dbe69e727416) luminous (stable)": 12
    },
    "mds": {},
    "overall": {
        "ceph version 12.2.4 (4832b6f0acade977670a37c20ff5dbe69e727416) luminous (stable)": 18
    }
}
 

mbaldini

Active Member
Nov 7, 2015
170
21
38
Thank you very much for the link, I had the same exact problem of the linked thread and in the 4th post I found the solution
Cephx is disabled so I removed the files in /etc/pve/priv/ceph , restarted pvestatd and pvedaemon and everything is back healthy

Problems solved!
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!