pvesm shows rbd error: rbd: listing images failed: (2) No such file or director

[immo]

root@prox22:~# pvesm list proxmox
rbd error: rbd: listing images failed: (2) No such file or directory

a second pool can be successfully asked

root@prox22:~# pvesm list data
Volid Format Type Size VMID
data:base-11000-disk-0 raw images 8589934592 11000
data:base-323-disk-0 raw images 68719476736 323
data:vm-100-disk-0 raw images 68719476736 100
data:vm-100-state-before_CDM_instalation raw images 5386188800 100
data:vm-100-state-before_LDAPS_Cert_import raw images 16650947584 100
...


rbd -p proxmox -m 10.160.253.101,10.160.253.105,10.160.253.107 -n client.proxmox_ceph --keyring /etc/pve/priv/ceph/proxmox.keyring ls
and
rbd -p data -m 10.160.253.101,10.160.253.105,10.160.253.107 -n client.proxmox --keyring /etc/pve/priv/ceph/data.keyring ls

works also without any issues

 

[mira]

Try `rbd ls -l`, which is used in `pvesm`. Provide its output here please.
Most likely there is a corrupted image or snapshot there which results in this issue.

 

[immo]

thisd looks crazy but my understanding is that in case just an external ceph cluster exists this is correct

ssh prox01 rbd ls -l
2022-08-31T17:48:16.115+0200 7f484a309340 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,: (2) No such file or directory
2022-08-31T17:48:16.115+0200 7f484a309340 -1 AuthRegistry(0x561d31c274a0) no keyring found at /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,, disabling cephx
2022-08-31T17:48:16.115+0200 7f484a309340 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,: (2) No such file or directory
2022-08-31T17:48:16.115+0200 7f484a309340 -1 AuthRegistry(0x7ffd2bf6f600) no keyring found at /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,, disabling cephx
2022-08-31T17:48:16.167+0200 7f484a309340 -1 monclient: authenticate NOTE: no keyring found; disabled cephx authentication
rbd: couldn't connect to the cluster!
rbd: listing images failed: (95) Operation not supported

----


cat /etc/ceph/ceph.conf
[global]
mon host = 10.160.253.107:6789, 10.160.253.105:6789, 10.160.253.101:6789

[client.proxmox]
rbd default data pool = data_ec
rbd_default_features = 13

[client.proxmox_ceph]
rbd default data pool = proxmox_ec
rbd_default_features = 13
root@prox01:~#

 

[mira]

Sorry, I wasn't clear enough, you still have to provide all additional parameters as you did when running `rbd ls`, but add `-l` at the end:
rbd -p proxmox -m 10.160.253.101,10.160.253.105,10.160.253.107 -n client.proxmox_ceph --keyring /etc/pve/priv/ceph/proxmox.keyring ls -l
and
rbd -p data -m 10.160.253.101,10.160.253.105,10.160.253.107 -n client.proxmox --keyring /etc/pve/priv/ceph/data.keyring ls -l
Otherwise the command can't connect to the ceph cluster and access the right pool.

 

[immo]

ok thanks a lot ...
so now we see errors on th proxmox rbd too

root@prox01:~# rbd -p proxmox -m 10.160.253.101,10.160.253.105,10.160.253.107 -n client.proxmox_ceph --keyring /etc/pve/priv/ceph/proxmox.keyring ls -l
rbd: error opening vm-350-disk-0: (2) No such file or directory
NAME SIZE PARENT FMT PROT LOCK
base-10001-disk-0 2.2 GiB 2
base-10001-disk-0@__base__ 2.2 GiB 2 yes
base-1002-disk-0 32 GiB 2
...
vm-998-disk-0 4 GiB 2
rbd: listing images failed: (2) No such file or directory

 

[mira]

Looks like there's some leftover VM disk still there, but with the backing objects missing.
Try removing it via `rbd`. After removing it you should no longer see this error.

 

[immo]

yeah ... I found on the cluster the still shown vm disk removed it vi rbd command at the cluster directly ... and since than.. The pvesm works again.