NFS Multipath (or at least failover)

[Kei]

Hello,
I'm reading that someone in the community does use NFS 4.1 with multipath, so I've tried configuring it but altought I can choose 4.1 or 4.2 as version, I can only select a single IP address. Could someone kindly explain how can I set this up? Thank you!

 

[Alwin]

The Proxmox VE NFS Plugin only allows one IP to be set. Best mount your multipath NFS separately and use it as a directory storage. You will need to set the shared flag as well.
https://pve.proxmox.com/pve-docs/pve-admin-guide.html#storage_nfs

 

[LnxBil]

It would be great if you report back on how you did it. I just read that is should be simple, but I've never tried it.

 

[Kei]

The Proxmox VE NFS Plugin only allows one IP to be set. Best mount your multipath NFS separately and use it as a directory storage. You will need to set the shared flag as well.
https://pve.proxmox.com/pve-docs/pve-admin-guide.html#storage_nfs

Hi Alwin, please correct if I'm wrong but the document you linked to does not explain how to set up multipath NFS. If I understand correctly, this must be done on the OS, much like you would to when implementing multipath iSCSI. I would really appreciate if you could give us some hint on how this should be done. Thank you!

 

[Alwin]

Hi Alwin, please correct if I'm wrong but the document you linked to does not explain how to set up multipath NFS.

Correct. This shows the options for the NFS storage plugin.

If I understand correctly, this must be done on the OS, much like you would to when implementing multipath iSCSI. I would really appreciate if you could give us some hint on how this should be done. Thank you!

Yes. But AFAIK no howto or documentation exists for a setup on Proxmox VE. I suppose many people switch to Ceph or similar storage when the need for HA NFS arises.

 

[LnxBil]

Yes. But AFAIK no howto or documentation exists for a setup on Proxmox VE. I suppose many people switch to Ceph or similar storage when the need for HA NFS arises.

I didn't even find a Debian specific documentation, just general stuff. The only information I found is, that it is very simple, if you have to correct infrastructure. Both modules exist on PVE, so it should be straight forward to use. Maybe you should try that?

If I understand pNFS correctly, just giving the correct version should automatically use pNFS and that is currently also implemented in the PVE GUI:

 

[Alwin]

If I understand pNFS correctly, just giving the correct version should automatically use pNFS and that is currently also implemented in the PVE GUI:

NFS with a failover IP should work with our plugin but multipath NFS (multiple IPs) will need a fstab entry.

 

[xcdr]

Is there any documented and supported configuration for this? How I can configure valid multipath setup?

Mount via directory is tested?

 

[Alwin]

Is there any documented and supported configuration for this? How I can configure valid multipath setup?

Failover/multipath NFS is a highly customized setup, we do not provide documentation nor a supported configuration.

Mount via directory is tested?

No.

 

[matt42]

Currently looking into this to avoid iSCSI. @xcdr @LnxBil did you get pNFS / NFS multipathing working with Proxmox?

 

[efeu]

I currently testing this. I got pNFS working (for a while) by doing it this way:

modprobe nfs_layout_nfsv41_files
modprobe nfs_layout_flexfiles

mount x.x.1.x:/some/path /some/other/path -o vers=4.1,minorversion=1,max_connect=16
mount x.x.2.x:/some/path /some/other/path -o vers=4.1,minorversion=1,max_connect=16
mount x.x.3.x:/some/path /some/other/path -o vers=4.1,minorversion=1,max_connect=16
mount x.x.4.x:/some/path /some/other/path -o vers=4.1,minorversion=1,max_connect=16

You can see in syslog/dmesg that it connects multiple times, but for noreason I see it drops using additional paths after a while and only using first path, after it has been working for a while and gives multiple performance. No messages in syslog nor dmesg. So pNFS is still buggy - sad.

 

[spirit]

I currently testing this. I got pNFS working (for a while) by doing it this way:

modprobe nfs_layout_nfsv41_files
modprobe nfs_layout_flexfiles

mount x.x.1.x:/some/path /some/other/path -o vers=4.1,minorversion=1,max_connect=16
mount x.x.2.x:/some/path /some/other/path -o vers=4.1,minorversion=1,max_connect=16
mount x.x.3.x:/some/path /some/other/path -o vers=4.1,minorversion=1,max_connect=16
mount x.x.4.x:/some/path /some/other/path -o vers=4.1,minorversion=1,max_connect=16

You can see in syslog/dmesg that it connects multiple times, but for noreason I see it drops using additional paths after a while and only using first path, after it has been working for a while and gives multiple performance. No messages in syslog nor dmesg. So pNFS is still buggy - sad.

I think it's also depend of the nfs server implementation.
It's working fine with netapp array for exemple.

 

[efeu]

Well after an unwanted switch power down, I dont see any path drops anymore. So I guess it's working as it should.