Operations Time Out

[adrian.hilt]

Hello, recently I have been facing an issue with all operations of proxmox. Whenever I try to do something I get a timeout.

TASK ERROR: clone failed: error during cfs-locked 'storage-freenasNFS' operation: unable to create image: got lock timeout

My storage is a freenas connected over NFS. Didn't have this problems until updating to 6.x, and there is not really a clue of why and when it started.

Migrations, clones, snapshots, backups, get timeout. In the case of a snapshop I also loose access to the vnc and the machines freeze.

I appreciate any help

 

[oguz]

hi,

most likely it could be network connectivity issues with your nas. is everything stored there? does it work if you move a VM to local and try there?

could you post pveversion -v and /etc/pve/storage.cfg ?

 

[adrian.hilt]

Hi, this is the output of the pveversion -v

root@proxmox2:~# pveversion -v
proxmox-ve: 6.2-1 (running kernel: 5.4.44-2-pve)
pve-manager: 6.2-10 (running version: 6.2-10/a20769ed)
pve-kernel-5.4: 6.2-4
pve-kernel-helper: 6.2-4
pve-kernel-5.3: 6.1-6
pve-kernel-5.4.44-2-pve: 5.4.44-2
pve-kernel-4.15: 5.4-12
pve-kernel-5.3.18-3-pve: 5.3.18-3
pve-kernel-5.3.13-1-pve: 5.3.13-1
pve-kernel-4.15.18-24-pve: 4.15.18-52
pve-kernel-4.15.18-15-pve: 4.15.18-40
pve-kernel-4.15.18-14-pve: 4.15.18-39
pve-kernel-4.15.18-13-pve: 4.15.18-37
pve-kernel-4.15.18-10-pve: 4.15.18-32
pve-kernel-4.15.18-9-pve: 4.15.18-30
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.0.4-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.16-pve1
libproxmox-acme-perl: 1.0.4
libpve-access-control: 6.1-2
libpve-apiclient-perl: 3.0-3
libpve-common-perl: 6.1-5
libpve-guest-common-perl: 3.1-2
libpve-http-server-perl: 3.0-6
libpve-storage-perl: 6.2-5
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.2-1
lxcfs: 4.0.3-pve3
novnc-pve: 1.1.0-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.2-9
pve-cluster: 6.1-8
pve-container: 3.1-12
pve-docs: 6.2-5
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-2
pve-firmware: 3.1-1
pve-ha-manager: 3.0-9
pve-i18n: 2.1-3
pve-qemu-kvm: 5.0.0-11
pve-xtermjs: 4.3.0-1
qemu-server: 6.2-11
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-1
zfsutils-linux: 0.8.4-pve1

And this is the /etc/pve/storage.cfg

root@proxmox2:~# cat /etc/pve/storage.cfg
dir: local
path /var/lib/vz
content iso,backup,vztmpl

lvmthin: local-lvm
thinpool data
vgname pve
content rootdir,images

nfs: freenasNFS
export /mnt/Storage
path /mnt/pve/freenasNFS
server 10.0.0.2
content images
options vers=3


Right now we think it could be the connectivity. Any recommendations on it?

 

[oguz]

hi,

where is the NAS located? is it on the same network? what are the normal latencies between host and NAS?

for example can you try manually copying a file to your NAS?

also my other question:

is everything stored there? does it work if you move a VM to local and try there?

 

[adrian.hilt]

Hi,
Yes the NAS is located on the same dedicated network. The latencies are small, an avg of 0.067 ms.
Everything is stored on the NAS.
Tried moving a VM to the local and I can do all the operations normally.