csf-lock storage-ceph-pool error: got lock request timeout

[n0bie]

helo
i face this kind of problem last year.. but still dont get any solution.
i have read this bug as well : https://bugzilla.proxmox.com/show_bug.cgi?id=1962
yes i am using module from modulegarden integrated with whmcs.

when daily cron running and terminate the VM, sometime i got storage-ceph-pool got lock.. and could not remove the disk:
ceph-pool: vm-xxx-disk-0
ceph-pool: vm-xxx-cloudinit

this makes the disk become orphan (disk that does not have parent )
so, to solved this we need to remove it manually.

the configuration of vm-xxx is removed (purging VM xxx from related configuration), but the disk still remains there.

this issue does not always appear, just some time in case if there are multiple VM need to be terminated.. then daily cronjob will do that in sequence.. first VM delete, second VM delete, and so on..
i notice that if there are multiple VM to be deleted, then this kind of issue will appear as well. the ceph-pool still in lock, so the process is still running until the related configuration purging.

do u have any suggestion about this case?
thank you

 

[dherzig]

Might be related to this here as well -- https://forum.proxmox.com/threads/e...mage-got-lock-timeout-aborting-command.65786/. How long does it take to destroy the machine? Maybe put some sleep into the sequence of machines to be deleted?

 

[n0bie]

yes maybe need to put some sleep? how long should be?

i notice about this option: Preallocation
in this case we use cluster ceph. where Preallocation option located?
and my case is not when create a new VM, but when destroy multiple VM in sequence.

 

[dherzig]

Preallocation will not be available for CEPH rbd, this is just an option for file-based storages [0] (I thought cephfs would be one, but I don't see this option for it neither, while on my NFS share I do).

According to the mentioned discussion, there's a 60s timeout for operations that acquire a cluster lock. Difficult to say, might go into some corosync latency or ceph timing topics. Have you already reached out to people at modulesgarden, as you're not running your own hardware? I just read over their offer and they do not dedicately talk about ceph and pve clustering (which in physical hardware would be quite demanding).

[0] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_common_storage_properties

 

[n0bie]

i am running on my own hardware.
using modulegarden module https://www.modulesgarden.com/products/whmcs/proxmox-ve-vps
yes i have talk with them, and they said the issue is about the proxmox itself: https://bugzilla.proxmox.com/show_bug.cgi?id=1962
but i think using some delay/sleep would be help.