cannot remove templates any longer after upgrade to pve 6.4

[rainer042]

This morning I upgraded my pve cluster (5 hosts) from 6.3 to PVE 6.4. Basically everything work fine except for one detail: I cannot delete VM templates. If I try I get an error message saying:

TASK ERROR: rbd error: rbd: listing images failed: (2) No such file or directory

My pve-version is: pve-manager/6.4-8/185e14db (running kernel: 5.4.119-1-pve)

The storage is an external ceph Nautilus cluster 14.2.16. If I run rbd ls -p pxa-rbd |grep 183 on a ceph node I see the template rbd: base-183-disk-0 where 183 is the id of the template. I was also able to delete this template file manually (snapshot and rbd) from the ceph side but still if I try to remove the template in proxmox I get the same error.

However if I run the rbd command below on one of my PVE servers I see the list of all rbds in the given ceph-pool pxa-rbd:

rbd -p pxa-rbd -m <list of monitors-ips> -n client.rz --conf /etc/pve/priv/ceph/ceph-pxa.conf --keyring /etc/pve/priv/ceph/ceph-pxa.keyring --auth_supported cephx
base-112-disk-0
base-129-disk-0
...

vm-100-disk-0
...

So access to ceph is working fine, but anyway pve tells me that listing rbd images fails.

base-183-disk-0 was initially also in this list until I manually removed it on the ceph side.

After upgrading to 6.3 I had a similar problem with templates see here:

https://forum.proxmox.com/threads/r...-storage-ceph-operation-rbd-snap-purge.82631/

the solution those days was to install a more recent ceph client on the pve hosts. At the moment there is a 14.2.19 ceph client installed from this repos:

deb http://download.proxmox.com/debian/ceph-nautilus buster main

So at the moment I cannot get rid of any template file since even when I manually delete the disk template on the ceph side pve is still unable to remove the VM from its config. Would it be safe to simply manually delete the config file for this template /etc/pve/qemu-server/183.conf ?

Any ideas?

Thanks
Rainer

 

[rainer042]

After more searching I found this posting:

https://forum.proxmox.com/threads/r...failed-2-no-such-file-or-directory-500.66866/

So I also looped over all rbds in the pool with the template I could not delete:

for i in `rbd -p pxa-rbd ls`; do echo "**** $i"; rbd -p pxa-rbd info $i; done

And there really was one rbd for which rbd info failed. After removing this rbd via

rbd rm  vm-179-disk-0 -p pxa-rbd

the problem was gone.

Thanks
Rainer