Cannot create/migrate disk into NFS storage

[Mikepop]

Hi, I got this error:

create full clone of drive scsi0 (ceph-vm:vm-108-disk-0)
TASK ERROR: storage migration failed: error with cfs lock 'storage-nfsvms': unable to create image: got lock timeout - aborting command

I've searched forum and only found a thread regarding quorum issues, not the case, I've tried this even with one node only up and two down and still get the timeout.

The strange thing is this only happends with big disk on migration, 1 Tb or more, I can migrate smaller disks withouts problems.
Other strange thing, if I try to create a disk on nfs storage I get same timeout even if the disks are smaller.

I had no problems before with bigs or small disks on NFS on this cluster and all this started after adding a new bridge over other netwok/interface in net config in the cluster nodes, it should not be related, but I don't know.

I've tried deleting the storage, create another one over another 10 Gb interface/network and same issue.
NFS is mounted from Freenas zfs share.

Is there any way to debug the disk creation on nfs storage or doing manually to see exactly what it's going on?

Regards

 

[Astraea]

After creating the new bridge did you restart the node?

 

[Mikepop]

After creating the new bridge did you restart the node?

Hi Astraea, thnaks for reply. Yes, I've updated and restarted all nodes.

Regards

 

[Astraea]

does removing the bridge fix the issue?

 

[Mikepop]

No, I've tried that too.
I've even see the disk in the nfs share content from gui created before the timeout lock when creating a 1 Tb vm disk, when error shows up the disk is deleted from nfs.

Regards

 

[rtetzloff]

Just wondering if anyone has been able to resolve this. I'm seeing the same thing:

Apr 29 19:25:45 pve-1 pvedaemon[11541]: <root@pam> starting task UPIDve-1:00006BC1:00842E58:5CC79609:qmcreate:102:root@pam:
Apr 29 19:26:00 pve-1 systemd[1]: Starting Proxmox VE replication runner...
Apr 29 19:26:01 pve-1 systemd[1]: Started Proxmox VE replication runner.
Apr 29 19:26:45 pve-1 pvedaemon[27585]: VM 102 creating disks failed
Apr 29 19:26:45 pve-1 pvedaemon[27585]: unable to create VM 102 - error with cfs lock 'storage-nas-1-vmpool': unable to create image: got lock timeout - aborting command
Apr 29 19:26:45 pve-1 pvedaemon[11541]: <root@pam> end task UPIDve-1:00006BC1:00842E58:5CC79609:qmcreate:102:root@pam: unable to create VM 102 - error with cfs lock 'storage-nas-1-vmpool': unable to create image: got lock timeout - aborting command

This same thing happens on both NFS or CIFS, seemingly affecting disks about 300-500GB in size or greater. FreeNAS server and Proxmox server are connected via 10GE with MTU 9000.

**EDIT**

Looks like this was resolved by setting ZFS sync on FreeNAS dataset to disabled. This caused a 1TB image to be created almost instantly (~5s). I was able to create a 4TB image in about 30 seconds this way. Pretty night and day difference.

 

[shardas]

Hi all,
got exact same issue here.

• Full clone / Migrate from Local to NFS -> failing
• Creating new VM directly on NFS -> succeed

does anyone resolved this?

 

[shardas]

If anyone should have the same issue, on my case the problem was CloudInit:

root@pve:~# qm clone 101 103 --full true --name VM02 --storage NFStorage
create full clone of drive ide0 (local-lvm:vm-101-cloudinit)
clone failed: error with cfs lock 'storage-Storage': unable to parse volume filename 'vm-103-cloudinit'

Remove the Cloudinit device and Cloning will work