unable to create image: got lock timeout - aborting command

[KenZ71]

Unable to create VM & CT, getting below error. It must be a silly user error - this used to work, I do have one other VM in the same pool

VM_Storage is on the ZFS Pool on the Proxmox host the folder has rwx permissions for owner, group, other - not ideal but trying to eliminate all causes.

Folder is owner by root - should it be owner by someone else?

TASK ERROR: unable to create VM 110 - error with cfs lock 'storage-VM_Storage': unable to create image: got lock timeout - aborting command

from syslog:

Sep 09 09:48:00 ml10 systemd[1]: Started Proxmox VE replication runner.
Sep 09 09:49:00 ml10 pvedaemon[10605]: <root@pam> starting task UPID:ml10:000044B7:0136B1B4:5D76584C:qmcreate:110:root@pam:
Sep 09 09:49:00 ml10 systemd[1]: Starting Proxmox VE replication runner...
Sep 09 09:49:00 ml10 systemd[1]: pvesr.service: Succeeded.
Sep 09 09:49:00 ml10 systemd[1]: Started Proxmox VE replication runner.
Sep 09 09:50:00 ml10 pvedaemon[17591]: VM 110 creating disks failed
Sep 09 09:50:00 ml10 pvedaemon[17591]: unable to create VM 110 - error with cfs lock 'storage-VM_Storage': unable to create image: got lock timeout - aborting command
Sep 09 09:50:00 ml10 systemd[1]: Starting Proxmox VE replication runner...
Sep 09 09:50:00 ml10 pvedaemon[10605]: <root@pam> end task UPID:ml10:000044B7:0136B1B4:5D76584C:qmcreate:110:root@pam: unable to create VM 110 - error with cfs lock 'storage-VM_Storage': unable to create image: got lock timeout - aborting command
Sep 09 09:50:00 ml10 systemd[1]: pvesr.service: Succeeded.

 

[KenZ71]

Odd, seems to only be an issue for VMs with larger drivers.

That is I can create a VM with same parameters but only 32GB disk. When I try with a 250GB disk it fails. I suppose I can simply write my data to the ZFS pool. Still odd though.

 

[malevolent]

Same here, with 2T+ disks it always fails. Creation of 4TB+ disk also fails, I need to create a 2TB disk and then grow it 2TB more.
But once created , the problem of moving them is here. I'm on Gluster over ZFS

 

[malevolent]

After making several changes on ZFS, it seems I have a combination of slow disks and gluster. Fast disks have no problem with ZFS + Gluster. Those slow disks and ZFS can also create/move huge disks. But when combining the slow disks with gluster, a timeout occurs.
I tried ZFS performance options (sync, ZIL, SLOG, etc) but the problem persists if I put the image on top of Gluster. I also tried gluster performance options (performance.stat-prefetch: off, performance.io-cache: off, performance.quick-read: off, performance.read-ahead: off) but there is no luck.
I think the only solution is to create smaller disks (512GB) and then "join" them inside the VM OS (ie.: Linux LVM or Windows Dinamic disk)