fail when trying to delete more than 1 vm or container at the same time

[pixel]

using ceph for storage, when trying to delete more than one container or vm at the same time, the task fails with (in the tasks log)

Error: unexptected status

details of that entry:

trying to aquire cfs lock 'storage-ceph' ...TASK ERROR: got lock request timeout

proxmox version 4.4, linux 4.4.35-1-pve x86_64
ceph version 0.80.7

ceph nodes runs ubuntu 16.04.1 lts, kernel 4.4.0-57-generic
ceph version 10.2.5

p.s. i like the new site design!

 

[fabian]

yes, such operations take a storage-wide lock (to prevent races and name conflicts). if the operations take very long (such as ceph volume deletes with lots of data), we rather fail and return than wait forever.

 

[pixel]

thanks. what other operations take storage-wide locks? is this lock on proxmox or ceph side?

 

[fabian]

thanks. what other operations take storage-wide locks? is this lock on proxmox or ceph side?

• check whether a volume is a still used base volume (linked clone / template)
• linked clone creation
• transforming a volume to a base volume (usually a rename, but can be more expensive depending on storage)
• volume creation
• volume deletion

note that for most storages all of the above should be quite fast, so the locking is not that problematic unless you make a lot of requests in parallel. I also noticed that it can take quite a while with ceph, compared to other storage types which return almost immediately and delete in the background / just mark as deleted.