VM Create Failing due to lock in CEPH.

[fat_kid]

Hey,
Has anyone seen this before? Sometimes we are seeing a VM create (or clone) fail due to the following error:

TASK ERROR: create failed - rbd error: got lock timeout

When this happened earlier today I could reproduce the issue until I ran some rbd commands to check that I could create or copy an image directly in Ceph.
After I ran these commands then Proxmox could create images again.
This is only happening intermittently.
Any help or insight on this would be appreciated.
Using Proxmox v3.3

 

[dietmar]

How many OSDs do you run with your ceph cluster?

 

[fat_kid]

Currently 84 OSD's in the cluster.

 

[dietmar]

Maybe there was some backfill in progress when that happened?

 

[fat_kid]

It looks like we're still getting this.

Maybe there was some backfill in progress when that happened?

I checked and there definitely wasn't a backfill running this time.

rbd ls -l <pool_name>

is what clears the lock.
It's a bit odd.

Any ideas welcomed.