Ceph unable to delete multiple VM disks at a time

[alext]

Hi guys,

I've got a 3-cluster Proxmox system with Ceph (...and loving it!)
When I delete multiple VM's in quick succession, the first VM's disk gets deleted (takes a few minutes), but the other VM's give the following error message about 20 times while the first one is being deleted.

"trying to acquire cfs lock 'storage-ceph1' ..."

After trying, it then gives up with the following message:
"error with cfs lock 'storage-ceph1': got lock request timeout"

I've been able to delete the disk files manually using the rbd commands, but do you know if this is a limitation of the Prox/Ceph system (i.e. you can only delete one VM at a time), or am I doing something wrong?

Running Proxmox 5.2-8 with Ceph 12.2.8 but I've seen the problem for the last couple of months.

Thanks!

Alex

 

[Alwin]

"trying to acquire cfs lock 'storage-ceph1' ..."

It works only one at a time.

 

[Alwin]

could you please open a bug-report

For?

 

[alext]

Hey Alwin,

Thanks for confirming.
Would indeed be good to have some sort of check beforehand that then doesn't allow a second VM to be deleted (or places it in a queue) while the other one is removed - to avoid the VM being deleted but not the disk.

 

[Alwin]

Would indeed be good to have some sort of check beforehand that then doesn't allow a second VM to be deleted

The lock does that already, doesn't it?

The disk deletion is now done with the command line tool and that doesn't return till the disk has been deleted. There are thoughts on changing this, but AFAIK no work has been done yet. You may open up a feature request in our bug tracker to have it written. https://bugzilla.proxmox.com/

 

[alext]

Well, yeah, it stops the disk being deleted, but it doesn't stop the VM from being deleted. The VM does get removed, even though the disk isn't. Not a dangerous scenario, but does mean that the disk can fill up with unused objects which cannot be deleted from the GUI.

Yeah, will create a feature request, thanks!