rbd error: rbd: listing images failed: (2) No such file or directory (500)

proxwolfe

Well-Known Member
Jun 20, 2020
526
61
48
50
Hi,

I have a three node PVE cluster with Ceph with (currently) three pools. This is about one of them. When I click in the GUI on one of the pools and want to view the VM Disks or CT Volumes, I just get the error message "rbd error: rbd: listing images failed: (2) No such file or directory (500)"

I have already tried to removed the configs of previously deleted VMs from all the nodes (that referenced drives in the offending pool and that had not been automatically removed when I deleted the VMs) which seems to have helped someone else in this situation. But for me, it did nothing. Still getting the error message.

When I do an rbd ls, I get a number of VM disks but before that it shows:

Code:
rbd: error opening vm-123-disk-0: (2) No such file or directory
rbd: error opening vm-120-disk-0: (2) No such file or directory
rbd: error opening vm-121-disk-1: (2) No such file or directory
rbd: error opening vm-110-disk-0: (2) No such file or directory
NAME                      SIZE    PARENT  FMT  PROT  LOCK
base-113-disk-0           15 GiB            2        excl
base-113-disk-0@__base__  15 GiB            2  yes       
base-113-disk-1           15 GiB            2        excl
base-113-disk-1@__base__  15 GiB            2  yes       
base-120-disk-1           15 GiB            2           
base-120-disk-1@__base__  15 GiB            2  yes       
base-121-disk-0           15 GiB            2           
base-121-disk-0@__base__  15 GiB            2  yes       
base-126-disk-0           15 GiB            2        excl
base-126-disk-0@__base__  15 GiB            2  yes       
base-135-disk-0           15 GiB            2           
base-135-disk-0@__base__  15 GiB            2  yes       
base-135-disk-1           15 GiB            2        excl
base-135-disk-1@__base__  15 GiB            2  yes       
base-400-disk-0            8 GiB            2           
base-400-disk-0@__base__   8 GiB            2  yes

And then I realized that the GUI does not show my VM templates anymore. TBH, I don't remember how many there were, but the list above looks about right.

At the same time, there are two or three VMs (and one CT) that had their virtual disks on that pool and that won't start because the disks apparently went missing. All other VMs that also had their virtual drives on this pool operated normally (I have since moved the disks to another pool).

So what do I need to do, in order to get the pool's contents shown in the GUI again? And where are the missing virtual drives?

Thanks!
 
So VMs 110, 120, 121 and 123 no longer exist, or do they? If so, please post the configs of the VMs with qm config VMID here.
 
@alexskysilk can you explain to me why such an error message can occur with an rbd ls? That just seems completely illogical to me, since rbd ls only outputs what actually exists. There is no rbd configuration file that says which images should exist or which should not.
 
@alexskysilk can you explain to me why such an error message can occur with an rbd ls?
It totally threw me for a loop when it happened to me, but its apparently possible to have the image on the file system, but because its invalid some file system utilities (eg, rbd ls) breaks on actually listing it. it will remove normally however, which is the only "fix" afaict