Adding NFS v4 Share to Proxmox Cluster Fails if NFS Server Does Not Support NFSv3 Connections: Looking to Confirm Behavior Before Reporting Bug

SInisterPisces

SInisterPisces

Active Member
Proxmox Subscriber
Sep 1, 2022
240
46
33
40
Hello,

I have an up-to-date TrueNAS SCALE bare metal server running that I'm using to, among other things, store LXC templates and ISOs for Proxmox. I initially configured TrueNAS's NFS service to disable NFS v3, as I knew Proxmox supported v4.2 and that's what I wanted to use.

However, Proxmox was only able to see the server, but not list exported shares of the NFS v4-only server. I had to re-enable the NFSv3 protocol on the server to get the share list working again so Proxmox could connect to it within the GUI, even when explicitly telling Proxmox to use NFS 4.2.

At first I thought I'd broken my TrueNAS server's NFS implementation somehow, and went on a fun troubleshooting journey and even filed my first TrueNAS bug report. See: https://forums.truenas.com/t/truenas-scale-dragonfish-24-04-1-1-nfs-service-not-working/7612

tl;dr NFS v4 does not support the showmount command. If Proxmox is relying on this command to list the available exported shares when adding an NFS share at the cluster level, it will fail if the server has NFS v3 disabled.

I wanted to confirm that is how PVE's web interface for adding NFS shares actually works. If it's not, I don't know what's causing this issue.

Also, while it's not a bug, it's definitely confusing if you're using NFS for the first time. Proxmox was the first time I needed to use an NFS share; I'd always used SMB before that, and disabling older versions of SMB doesn't impact the functionality of the version of SMB you're actually using. That said, I'm considering filing a bug to request a help popup/expansion somewhere appropriate in the Add NFS dialogue box to indicate that NFS is required. I spent nearly 3 hours troubleshooting an NFS server that was working correctly and would like to find a way to help other people not do that.
 
  • Like
Reactions: Bingo600
Unfortunately pve is still relying on the showmount nfs3 rpc calls even if all nfs mounts are and even default to 4.2 and is still a pity. Hopefully that will be fixed in a future release eg. by communicating over the cluster service for export informations.
 
  • Like
Reactions: SInisterPisces
waltar said:
Unfortunately pve is still relying on the showmount nfs3 rpc calls even if all nfs mounts are and even default to 4.2 and is still a pity. Hopefully that will be fixed in a future release eg. by communicating over the cluster service for export informations.
Click to expand...
Is there a proof concept for how the cluster service could be used to address this?

A bare-metal TrueNAS server wouldn't be part of the cluster. So, would the cluster service, hypothetically, be in charge of reaching out to the TrueNAS server, enumerating its NFS shares via a temporary NFSv3 connection, and sending that information somewhere that the PVE storage manager could use it to present the kind of interface we currently have in the GUI?

(And yes, I'm being vague because I don't fully understand how any of those subsystems work...just enough to do the things I actually need to do so far. :) )
 
BobhWasatch said:
I use FreeIPA so I export automout maps and use Kerberos to manage authentication. For most people that is a lot of infrastructure to get trained up on. FreeIPA simplifies it a lot but still ..
Click to expand...
I'm glad you found a solution, but it would absolutely be a feature regression to see something this complex in Proxmox. I don't even know what FreeIPA is. I'm assuming you didn't wave a beer at it until it complied.

Right now, we point PVE at an NFS server, PVE figures out what shares it can see, and asks us which one we want. The learning curve is minimal. I know because I managed to add two NFS shares and actually use them, which is ... still a bit surprising, to be honest. :P

Any replacement for NFSv3's RPC functionality has to be at least as simple and easy to use if it's going to be user facing.
 
Pve should not known any exports which it hasn't configured and with "mountpoint -q or -d </mountdir>" you are able to validate if the mount is active or dead. "df" might hang short or long, "mount" show died mount but status unknown.
That should be enough to know for pve and the usage of showmount and rpcinfo could gone ... :)
 
Last edited:
  • Like
Reactions: SInisterPisces and esi_y
SInisterPisces said:
I'm glad you found a solution, but it would absolutely be a feature regression to see something this complex in Proxmox. I don't even know what FreeIPA is. I'm assuming you didn't wave a beer at it until it complied.
Click to expand...
FreeIPA is basically Kerberos + LDAP + PKI integrated together. Roughly like Active Directory for Linux.

SInisterPisces said:
Right now, we point PVE at an NFS server, PVE figures out what shares it can see, and asks us which one we o want. The learning curve is minimal. I know because I managed to add two NFS shares and actually use them, which is ... still a bit surprising, to be honest. :p

Any replacement for NFSv3's RPC functionality has to be at least as simple and easy to use if it's going to be user facing.
Click to expand...
There is no replacement for RPC in NFSv4. Which means no share browsing functionality, at least not that I am aware of. You have to set up an out-of-band way to obtain that information and I don't think there are any standards in that area.

The main reason to use NFSv4 over v3 are TCP instead of UDP and security. There are others, such as better file locking semantics and user ID mapping. Those by themselves might be worth having but TCP and security are the reason v4 exists.

Many (most?) NFSv3 implementations already support TCP so that's not really a differentiator any more. The security part still is.

NFSv4 no longer places almost complete trust in the client machines. You can configure NFSv4 to cryptographically authenticate both client machines and users. It is then much more difficult for someone with root on a client to read arbitrary user data. NFSv4 can also send signed and/or encrypted messages over the wire to prevent man-in-the middle attacks and snooping.

Problem is, to actually use those features requires Kerberos on the server and all of the clients. Kerberos is not trivial to set up. Which is why I use FreeIPA.

If you aren't willing to set that up then NFSv4 has only limited advantages over NFSv3 with TCP. They might be really important for some people, but for sharing a volume to put your VM images on over a private network probably don't matter all that much.
 
Last edited:
But nfsv4 has server side not this security part if you export to a network segment and must have nfs v3 enabled same time so that any other manipulated client can mount either without the security features enabled - that's the point.
 
waltar said:
But nfsv4 has server side not this security part if you export to a network segment and must have nfs v3 enabled same time so that any other manipulated client can mount either without the security features enabled - that's the point.
Click to expand...
That is true. Best practice would be to export different shares via NFSv3 and v4 if you need to have v3 enabled.

My point was that NFSv4 simply doesn't support the share browsing features that the OP was asking the Proxmox team to implement. That only works with v3, or with v4 + LDAP or something like that. Since there aren't any real standards for the latter it isn't surprising the PVE doesn't implement it.

And saying "v4 has better security" is only true if you go to some trouble to make it so.
 
BobhWasatch said:
NFSv4 simply doesn't support the share browsing features that the OP was asking the Proxmox team to implement.
Click to expand...

I do not really like it myself when I ask for a feature and someone asks me why I need it, as if implying I should not, but I am just curious, why does someone need to browse the shares, as in, I am genuinely curious.
 
  • Like
Reactions: waltar
I did want NFS v4's enhanced file locking features, as I'm a bit paranoid about screwing up and telling two endpoints to try to manipulate the same data. Coming from a very low powered QNAP box, I'm also in the habit of turning off services I'm not using to save on resources and decrease attack vectors. So, I turned off NFSv3 and spent several hours trying to figure out how I broke NFSv4 share enumeration becasue I didn't realize at the time they were interconnected. Oops.

esi_y said:
I do not really like it myself when I ask for a feature and someone asks me why I need it, as if implying I should not, but I am just curious, why does someone need to browse the shares, as in, I am genuinely curious.
Click to expand...
In Proxmox, when you use the Datacenter --> Storage GUI to add an NFS share as storage for your cluster, it's using the NFSv3 showmount feature to fetch a list of exported NFS shares from the server you're trying to connect to--even if those shares are only available via NFSv4.

If you disable NFSv3 on your storage server, the Proxmox GUI can't see the NFSv4 shares, so it can't connect to them via the GUI.
So, if you actually wanted to disable NFSv3 on the server and still use NFSv4 shares in Proxmox, I assume you'd need to do some sort of CLI-based stuff with pvesm and tell it exactly how to find and connect to the share (network path, password, etc.)

When I started this thread, as I noted at the end of my OP, I wasn't asking for a functionality change, but a documentation update. As annoying as it is, NFSv4 is working as intended here. I was considering filing a bug report/feature request to get the actual GUI or help popup changed to remind the user that the share discovery feature in the Proxmox GUI requires NFSv3 to be enabled server-side, which is not obvious if you're brand new to NFS. (And yes, I realize that Proxmox VE is a commercial-focused product, so there's a balance to be struck between documentation that's good for trained professional users and documentation that's good for home server enthusiasts trying to teach themselves things like NFS as they go.)

FreeIPA is basically Kerberos + LDAP + PKI integrated together. Roughly like Active Directory for Linux.
Click to expand...
@BobhWasatch , thanks for this explanation. I'll have to look into this to see if it'd be useful on my network. The whole NFS thing is a minor annoyance, and I've already got six unfinished projects in the air, so I'm not sure I'm ready to implement another security layer right now; I have a sneaking suspcion I'd have to redo a lot of stuff that works well enough for the moment. ;)
 
  • Like
Reactions: waltar
SInisterPisces said:
In Proxmox, when you use the Datacenter --> Storage GUI to add an NFS share as storage for your cluster,
Click to expand...
I will focus only on this part, not to get off track. If I omitted something important, feel free to let me know bluntly.

SInisterPisces said:
it's using the NFSv3 showmount feature to fetch a list of exported NFS shares from the server you're trying to connect to--even if those shares are only available via NFSv4.
Click to expand...
Not if you select v4 (see also my early post above), but this is not what got me puzzled now.

SInisterPisces said:
If you disable NFSv3 on your storage server, the Proxmox GUI can't see the NFSv4 shares, so it can't connect to them via the GUI.
Click to expand...

Here's the thing, I have an NFS server with /proc/fs/nfsd/versions showing -3 +4 +4.1 +4.2. In fact I masked rpcbind.service as well. I go ahead in PVE GUI as you described in the Datacenter and select NFSv4 instead of Default and mount my export just fine via the said GUI. True, if I left it as default, it says it's not online (apparently on the v3).

SInisterPisces said:
share discovery feature in the Proxmox GUI requires NFSv3 to be enabled server-side
Click to expand...

if you mean mountd dependent "discovery" then yes you may ask for such reminder, but I am not sure how this worked on NFSv3 shares, how does one show showmount -e in the GUI? (If I sound stupid not knowing this, it is because I am not all that familiar with the GUI itself, but that's why I tested it for this time as mentioned above.)

EDIT: So the short answer to this ...
SInisterPisces said:
tl;dr NFS v4 does not support the showmount command. If Proxmox is relying on this command to list the available exported shares when adding an NFS share at the cluster level, it will fail if the server has NFS v3 disabled.
Click to expand...

No it does not rely on it.
 
Last edited:
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom