Best practise sharing NFS to proxmox cluster?

[Dirky_uk]

Hi, I have a small cluster with 3 nodes, configured for HA.
Most of my data is on the same network as the cluster, on a Drobo connected with thunderbolt to a Mac, running Catalina. (Yeah, ewwww I know)

I was planning on moving my services off the Mac to begin with, over to the cluster, and I will share the data from the Mac with nfsd.

My question is which option is best.
1) Mount the NFS volumes to the host OS of each proxmox node and pass this through to the VM / LXC
2) Mount the NFS shares direct to the VM's / LXC?

Any thoughts?
Thanks!
D

 

[VictorSTS]

For VMs, mount the NFS shares from the guest OS.
For LXCs, manually mount the NFS share in the Proxmox host using something like this [1] (option 3). Avoid mouting NFS inside the LXC as it may cause I/O deadlocks.

[1] https://forum.proxmox.com/threads/tutorial-mounting-nfs-share-to-an-unprivileged-lxc.138506/

 

[Dirky_uk]

For VMs, mount the NFS shares from the guest OS.
For LXCs, manually mount the NFS share in the Proxmox host using something like this [1] (option 3). Avoid mouting NFS inside the LXC as it may cause I/O deadlocks.

[1] https://forum.proxmox.com/threads/tutorial-mounting-nfs-share-to-an-unprivileged-lxc.138506/

Thanks! I appreciate the tips.

 

[dlasher]

For LXCs, manually mount the NFS share in the Proxmox host using something like this [1] (option 3). Avoid mouting NFS inside the LXC as it may cause I/O deadlocks.

[1] https://forum.proxmox.com/threads/tutorial-mounting-nfs-share-to-an-unprivileged-lxc.138506/

This approach has multiple disadvantages, not the least of which is performance reductions, since the bind mounts all share the host NFS mount (in addition to LACP-bundles being single-member utilized).

Anyone figure out how to fix the io deadlock issues?

EDIT:

One potential note -- I was using "pct stop" to halt the containers -- the following appears to work better - work in progress

pct shutdown ${CONTAINER_ID} --forcestop 1 --timeout 60

 

[VictorSTS]

This approach has multiple disadvantages, not the least of which is performance reductions, since the bind mounts all share the host NFS mount (in addition to LACP-bundles being single-member utilized).

I haven't used it much and performance was good enough for the apps that used it in a 4 node cluster. It got migrated to Ceph like 3 years ago. Of course might not be good enough for some use case. You can't really use LACP with NFS easily. An option could be using NFS 4.x multipath, another is to define more than one NFS storage to different NFS server IP's and spread your VM/LXC among them but can get cumbersome to maintain.

BTW, pct stops kills the CT, shutdown does an ordered shutdown of the services, then stops the LXC. How is that related to NFS?