NFS multipath

B

bluechris

New Member
Apr 8, 2024
5
0
1
Guys i need some help here. I searched in forum and i found some posts that people are able to connect with 2 paths with NFS but the instructions are not clear.
I have 2 truenas servers. One for vm's and one for backup. Both have 2x25gbit cards.
In each one i have setup different subnets in each card.
For example
10.40.0.18/24 1st card
10.50.0.18/24 2nd card

Now all the proxmox servers have the same configuration.
For example
10.40.0.1/24 1st card
10.50.0.1/24 2nd card

All proxmox servers and truenas are connected to the 1st Switch (Unifi Agg Pro) with their 1st card and with the 2nd card to my 2nd Unifi Agg Pro.
The 2 Unifi Agg Pro switches are interconnected with 2x25gbit with LACP.

So its obvious what i am trying to implement here, mclag without mclag ability in the switches but the above connection scenario works.
The problem relies in the NFS from proxmox side where no matter if i choose nfs 4.1, the connection is single and if i take out the one cable from a server , the nfs is lost.
I need from each server to connect to the truenas with dual path and in my example to be able to put in each proxmox to connect to 10.40.0.18 and 10.50.0.18 for every NFS share

Can someone help here?
Thanks
Chris
 
You need to bond (active+backup or lacp mode while it would effect to the same here) the 2 network cards of each nas server and get just 1 IP on the bond interface. Your proxmox servers even need a similar bonded config with their 1 IP. After that you can plugout a cable or shutdown a network interface or power off a switch and your nfs traffic would still be ongoing.
 
waltar said:
You need to bond (active+backup or lacp mode while it would effect to the same here) the 2 network cards of each nas server and get just 1 IP on the bond interface. Your proxmox servers even need a similar bonded config with their 1 IP. After that you can plugout a cable or shutdown a network interface or power off a switch and your nfs traffic would still be ongoing.
Click to expand...
This is not possible i think or i don't know it because LACP needs to be on the same switch. As i said the 2 switches dont have MCLag ability so in my mind this is not working. It works only if every server connects to 1 switch with no matter the way but in a case of failure or update one of the 2 switches there are 50% to loose the NFS, thats why i need to implement nfs4.1 with dual path and 2 subnets.

Many people here have achieve this in the forum, it just noone said exactly the steps and i dont have this knowledge.
I tried to put in fstab this

Code: 
10.40.0.18:/ProxmoxTrueNas2VmData /mnt/Pool3/ProxmoxTrueNas2VmData nfs vers=4.1,max_connect=4 0 2
10.50.0.18:/ProxmoxTrueNas2VmData /mnt/Pool3/ProxmoxTrueNas2VmData nfs vers=4.1,max_connect=4 0 2

but nothing changed, the truenas shows one connection only from the proxmox server.

In storage.cfg in the cluster what matter this connection is that

Code: 
nfs: ProxmoxTrueNas2VmData
    export /mnt/Pool3/ProxmoxTrueNas2VmData
    path /mnt/pve/ProxmoxTrueNas2VmData
    server 10.50.0.18
    content iso,images
    options nconnect=4,vers=4.1
    preallocation off
    prune-backups keep-all=1
 
Last edited:
bluechris said:
The 2 Unifi Agg Pro switches are interconnected with 2x25gbit with LACP.
Click to expand...
2 cables on each host "active/backup" mode bonded, 1 cable/switch, 1IP/host. That works with or without your lacp connection between your 2 switches.
 
waltar said:
2 cables on each host "active/backup" mode bonded, 1 cable/switch, 1IP/host. That works with or without your lacp connection between your 2 switches.
Click to expand...
I will try it thx even though this will not use the full bandwidth of the 2 links which is something extra that i want but its fine.
 
Mostly you would not get double bw with 2 switches as just seen by customer shortly before.
Lacp works for multiple cables to same switch if switch support that. So if you want 50Gb you would need 4 cables/host (and the number of switch ports)..
 
  • Like
Reactions: bluechris
floh8 said:
There 2 possibilities to achieve this goal. The better one is this: Netapp example
The hint for Link aggregation can your one skip. This is not best practise.
Click to expand...
I saw that and i tried it but i altered only the fstab in one host and now that i see it again i i need to do the same fstab edit in all hosts for this to work.
I will try it again later tonight

You know, if i achieve this, the backup time will also be reduced since it reads 6ΤΒs from the 1st truenas and writes to the second.
 
You must log in or register to reply here.
Top Bottom