Mount NFS 4.1 Share

[mschau]

Hi,

I cannot mount an NFS 4.1 Share via Webinterface - only NFS v3 works.

"create storage failed: error with cfs lock 'file-storage_cfg': storage 'NFS02' is not online (500)"

Mounting it on the same host via CLI works
nfsstat -m
/mnt/nfstest from server:/share
Flags: rw,relatime,vers=4.1,rsize=65536,wsize=65536,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=null,clientaddr=xx.xx.xx.xx,local_lock=none,addr=xx.xx.xx.xx

but:
root@VIRZ01-102:/mnt/nfstest/images# ls -la
total 16
drwxr-xr-x 4 nobody 4294967294 4096 Apr 17 08:42 .
drwxr-xr-x 5 nobody 4294967294 4096 Apr 16 15:25 ..
drwxr----- 2 nobody 4294967294 4096 Apr 16 16:27 100
drwxr----- 2 nobody 4294967294 4096 Apr 17 08:42 101



proxmox-ve: 5.1-42 (running kernel: 4.13.16-2-pve)
pve-manager: 5.1-49 (running version: 5.1-49/1e427a54)
pve-kernel-4.13: 5.1-44
pve-kernel-4.13.16-2-pve: 4.13.16-47
pve-kernel-4.13.13-2-pve: 4.13.13-33
corosync: 2.4.2-pve3
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.0-8
libpve-apiclient-perl: 2.0-4
libpve-common-perl: 5.0-30
libpve-guest-common-perl: 2.0-14
libpve-http-server-perl: 2.0-8
libpve-storage-perl: 5.0-18
libqb0: 1.0.1-1
lvm2: 2.02.168-pve6
lxc-pve: 3.0.0-2
lxcfs: 3.0.0-1
novnc-pve: 0.6-4
proxmox-widget-toolkit: 1.0-14
pve-cluster: 5.0-24
pve-container: 2.0-21
pve-docs: 5.1-17
pve-firewall: 3.0-7
pve-firmware: 2.0-4
pve-ha-manager: 2.0-5
pve-i18n: 1.0-4
pve-libspice-server1: 0.12.8-3
pve-qemu-kvm: 2.11.1-5
pve-xtermjs: 1.0-2
qemu-server: 5.0-24
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3
zfsutils-linux: 0.7.7-pve1~bpo9



Any idea on that?

 

[Alwin]

By default a connection to a NFS storage is done with NFSv3 protocol. Change the option in /etc/pve/storage.cfg (options vers=4).

 

[mschau]

Thanks for your reply.

I already created the entry in the storage.cfg manually - since the NFS Share couldn't be configured via the Webinterface.
The NFS Share appears in the Webinterface, but is not "active" and I can't access the storage via Webinterface (is not online, 500).

nfs: NFS01
export /NFS
path /mnt/pve/NFS01
server NFS-Server
content images,backup,iso
maxfiles 3
options vers=4.1 (>> already tried many different options here)

Like in my posting above described I already mounted the NFS Share via CLI to /etc/pve/NFS01 -> I can access the share via CLI (but the access rights seems odd -> "nobody" -> see posting above).

 

[Alwin]

options vers=4.1 (>> already tried many different options here)

Try with vers=4 and check how the nfs server exports it (pvesm nfsscan <server>).
https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_nfs_backend

Like in my posting above described I already mounted the NFS Share via CLI to /etc/pve/NFS01 -> I can access the share via CLI (but the access rights seems odd -> "nobody" -> see posting above).

You mean hopefully /mnt/pve/NFS01?

 

[mschau]

On the NetApp side I've temporarily deactivated NFS3 +4 - only 4.1 is active.

pvesm nfsscan netapp-nfs01
clnt_create: RPC: Program not registered
command '/sbin/showmount --no-headers --exports xx.xx.xx.xx' failed: exit code 1

You mean hopefully /mnt/pve/NFS01?

yes - the subfolders of the NFS01 share.

root@VIGT01-101:/mnt/pve/NFS01/images# ls -l
total 8
drwxr----- 2 nobody 4294967294 4096 Apr 16 16:27 100
drwxr----- 2 nobody 4294967294 4096 Apr 17 08:42 101

 

[Alwin]

These settings are up to your configuration on the NetApp.

 

[Whatever]

I'm facing "NFS storage is not online (500)" very often with NFS version >=4. However disk images are always creating without errors and all VMs work without any problem (pvesm only reports that NFS share is not online)

 

[czechsys]

Proxmox require nfs v3 working on nfs server side even if you want nfs v4+ only.

 

[Alwin]

Proxmox require nfs v3 working on nfs server side even if you want nfs v4+ only.

No. The nfs client is default debian package and it just requires correct setup.

 

[mschau]

Ok - seems that I got it running:

There were two problems - on NetApp side I had to adjust the permissions on the root path and on the share.
Second I had to add the "nolock" option in the manually created "storage.cfg":
For multipathing I created two entries for the same NFS share - but I have to test it if that really works...

ir: local
        path /var/lib/vz
        content iso,backup,vztmpl

lvmthin: local-lvm
        thinpool data
        vgname pve
        content images,rootdir

nfs: NFS01
        export /V_KVM_0000
        path /mnt/pve/NFS01
        server NFS-Server
        content images,backup,iso
        maxfiles 3
        options vers=4.1,nolock

nfs: NFS01
        export /V_KVM_0000
        path /mnt/pve/NFS01
        server NFS-Server
        content images,backup,iso
        maxfiles 3
        options vers=4.1,nolock

 

[czechsys]

No. The nfs client is default debian package and it just requires correct setup.

From when? Pvesm is scanning nfs storage via nfsv3 to see exports...or storage availability is via nfs3...Any updated docs about this?

 

[Alwin]

 my $cmd = ['/sbin/showmount',  '--no-headers', '--exports', $server];

We use the showmount command to get exports, so any protocol support by it, works. By default we set the vers to 3, to stay compatible with most installations out there.

 

[mschau]

Is there any experience for nfs multipath / clientid trunking?

I've testet my actual configuration (2x 10Gbit LACP) by setting up multiple VM's or copying files from/to the nfs share on my hosts, but there's only one NIC in use.

 

[Alwin]

It depends on the bond mode.
https://en.wikipedia.org/wiki/Link_aggregation
https://wiki.linuxfoundation.org/networking/bonding

 

[mschau]

I know - I thought with this configuration it should work:

auto bond0
iface bond0 inet static
 address  10.1.66.21
 netmask  255.255.255.0
 slaves eno10 eno11
 bond_miimon 100
 bond_mode 802.3ad
 bond_xmit_hash_policy layer3+4
# Storage