Proxmox VE 9.2.6 - NFS storage is reported as inactive although the mount is accessible

Klitschko

New Member
Dec 29, 2024
7
2
3
Summary
Proxmox reports an NFS storage as inactive even though the export is mounted and accessible from the host. The behaviour was observed with both NFSv4.x and NFSv3. NFSv4.x showed a GETATTR hang during investigation, while NFSv3 avoids that hang, but Proxmox still marks the storage as inactive.


Environment​

ProxmoxProxmox VE 9.2.6; Linux kernel 6.17.13-7-pve; host IP <management-IP>
Storage serverTrueNAS SCALE 25.02.x; management IP <management-server-IP>; storage IP <storage-server-IP>
NetworkSeparate management and storage VLANs; routing via firewall/router; connectivity verified
Other servicesA Proxmox Backup Server on the same storage host works without issue

Problem​

After adding the NFS storage in Proxmox, the storage is shown as inactive and Proxmox reports:

got timeout
unable to activate storage '<storage>' - directory '/mnt/pve/<storage>' does not exist or is unreachable

This is inconsistent with the actual state on the host, because the mount is present and readable.

Storage configuration​

The storage is configured as:

nfs: <storage-name>
export /mnt/<pool>/proxmox_backup
path /mnt/pve/<storage-name>
server <storage-server>
options vers=3

The same general behaviour was also observed with NFSv4.2.

Verified tests​

showmount​

This command returns immediately:

showmount --no-headers --exports <storage-server>

Observed runtime: about 3 ms.

The export list returned by the server includes the storage export and one additional legacy export. The additional export details are omitted here because they are not relevant to the issue.

rpcinfo​

rpcinfo -p <storage-server>

This works correctly.

Manual NFS mount​

A manual NFSv3 mount succeeds:

mkdir -p /mnt/testv3
mount -t nfs -o vers=3 <storage-server>:/mnt/<pool>/proxmox_backup /mnt/testv3

Directory access​

Both of the following work:

ls -la /mnt/testv3
ls -la /mnt/pve/<storage-name>

The directory contents are listed correctly.

stat()​

stat /mnt/pve/<storage-name>

This returns immediately.

pvesm​

pvesm status

This reports the storage as inactive.

inactive

pvestatd​

systemctl restart pvestatd

Restarting the service does not change the behaviour.

strace​

Tracing pvesm list shows that Proxmox internally executes:

/sbin/showmount --no-headers --exports <storage-server>

That command succeeds. The storage is still reported as inactive afterwards.

Additional investigation​

During earlier debugging of the NFSv4 storage, a blocked kernel stack showed:

rpc_wait_bit_killable()
rpc_execute()
rpc_run_task()
nfs4_do_call_sync()
nfs4_proc_getattr()
nfs_revalidate_inode()
nfs_getattr()
vfs_statx()

This indicated a GETATTR hang on the NFSv4 path.

Using NFSv3 avoids that GETATTR hang, but Proxmox still reports the storage as inactive even though the mount is readable.

Expected behaviour​

Since the export is reachable, showmount succeeds, rpcinfo succeeds, the filesystem is mounted, directory listing works, and stat() works, I would expect the storage to be reported as active.

Actual behaviour​

The storage is consistently reported as inactive with the message:

unable to activate storage
directory does not exist or is unreachable

Although the mount is present and fully accessible.

Question​

Is this a known issue or regression in the NFS storage handling or pvestatd in Proxmox VE 9.2.6?

If not, which additional checks does pvestatd perform after a successful mount that could still cause the storage to be marked as inactive?
 
I dont think its known issue.
Did you observed the same issue with kernel 7? e.g 7.0.14-8-pve
Also you could attempt older kernel 6.14.
 
Thanks for sharing.

Can you double check that your storage.cfg export entry does not contain a trailing /?
There is this bug report Bugzilla 6050 which could be related.

If you want to poke around in the code pvestatd uses the functions in NFSPlugin.pm to determine the status.
 
I dont think its known issue.
Did you observed the same issue with kernel 7? e.g 7.0.14-8-pve
Also you could attempt older kernel 6.14.
Thank you for the suggestion.

Unfortunately I cannot test with an older kernel at the moment because this is my production homelab. However, I can schedule a maintenance window and test both kernel 6.14 and 7.0.14-8-pve if this would help narrow down the issue.
 
  • Like
Reactions: longer
Thanks for sharing.

Can you double check that your storage.cfg export entry does not contain a trailing /?
There is this bug report Bugzilla 6050 which could be related.

If you want to poke around in the code pvestatd uses the functions in NFSPlugin.pm to determine the status.
Thank you for your reply.
I checked the configuration again.
The export entry does not contain a trailing slash.
It is configured as:

export /mnt/HDD-Pool/Proxmox-BackUp


I have also verified that:
  • showmount --no-headers --exports <server> completes immediately.
  • Manual NFS mounts work.
  • The mounted directory is readable (ls, stat).
  • Nevertheless, pvestatd still reports the storage as inactive.

I'll also have a look at Bugzilla #6050. Thank you for pointing me to it.
 
Thank you for your reply.
I checked the configuration again.
The export entry does not contain a trailing slash.
It is configured as:

export /mnt/HDD-Pool/Proxmox-BackUp


I have also verified that:
  • showmount --no-headers --exports <server> completes immediately.
  • Manual NFS mounts work.
  • The mounted directory is readable (ls, stat).
  • Nevertheless, pvestatd still reports the storage as inactive.

I'll also have a look at Bugzilla #6050. Thank you for pointing me to it.
I have verified the suggestion regarding the trailing slash.

There is no trailing slash in the export path.

Additionally:

  • showmount --no-headers --exports <server> completes in ~3 ms.
  • Manual NFS mount works.
  • ls and stat on the mounted directory work.
  • pvesm status still reports the storage as inactive.
 
Thanks for sharing.

Can you double check that your storage.cfg export entry does not contain a trailing /?
There is this bug report Bugzilla 6050 which could be related.

If you want to poke around in the code pvestatd uses the functions in NFSPlugin.pm to determine the status.
I noticed there are recent patches for NFSPlugin.pm (rpcinfo fallback / check_connection improvements).
Could this issue be related to the current check_connection() implementation in Proxmox VE 9.2.6?
 
I checked the git log and couldn't really find any changes that would break your setup, but feel free to link the commit you think might have broken your configuration.

Let's try the following and see if that helps:
  • Remove the storage configuration for the NFS share completely. Using the WebUI or deleting it from the `storage.cfg` in /etc/pve
  • Remove all mounts, the ones you create manually and the ones that PVE created for you. My suspicion currently is that you might have active mounts that interfere with the NFS mount of truenas.
    Code:
    umount -l /mnt/pve/<storage-name>
  • To verify if removal worked use:
    Code:
    cat /proc/mounts
    you can also grep the path you used in umount
    Code:
    cat /proc/mounts | grep /<path>
    I would recommend using the cat command to check what is currently mounted to make sure there are no stale entries that could be interfering.
After this you can use the WebUI to configure the NFS mount again. Make sure to check which version of NFS is configured on TrueNas and match it.

Please let me know if this helped.
 
I checked the git log and couldn't really find any changes that would break your setup, but feel free to link the commit you think might have broken your configuration.

Let's try the following and see if that helps:
  • Remove the storage configuration for the NFS share completely. Using the WebUI or deleting it from the `storage.cfg` in /etc/pve
  • Remove all mounts, the ones you create manually and the ones that PVE created for you. My suspicion currently is that you might have active mounts that interfere with the NFS mount of truenas.
    Code:
    umount -l /mnt/pve/<storage-name>
  • To verify if removal worked use:
    Code:
    cat /proc/mounts
    you can also grep the path you used in umount
    Code:
    cat /proc/mounts | grep /<path>
    I would recommend using the cat command to check what is currently mounted to make sure there are no stale entries that could be interfering.
After this you can use the WebUI to configure the NFS mount again. Make sure to check which version of NFS is configured on TrueNas and match it.

Please let me know if this helped.
Thanks for your help. I followed your suggestion and completely removed both old NFS storage definitions as well as all remaining/stale mount points under /mnt/pve.


During further troubleshooting I found that the Proxmox host itself did not have an IP address in my storage VLAN (VLAN 70). Therefore traffic from the Proxmox host (10.0.10.10) to the TrueNAS NFS address (10.0.70.10) was being routed through my OPNsense firewall instead of staying directly within the storage VLAN.

I have now added a dedicated VLAN interface on the Proxmox host:
vmbr1.70 -> 10.0.70.12/24
TrueNAS storage interface:
10.0.70.10/24
The NFS storage is now mounted directly over this network using NFS 4.1.
This made a very significant difference. Previously, even simple metadata operations could hang for 1–3 minutes and larger writes could stall completely. After adding the dedicated storage VLAN interface:
touch: ~0.034 s
rm: ~0.013–0.023 s

A 1 GiB NFS write test with conv=fsync now completes successfully:
1073741824 bytes copied, 3.85 s, 279 MB/s
iperf3 between Proxmox (10.0.70.12) and TrueNAS (10.0.70.10) also reaches approximately 2.35 Gbit/s in both directions.

So at this point the NFS issue appears to be resolved. The main problem seems to have been the network path rather than the NFS storage configuration itself: Proxmox was accessing the TrueNAS storage network through the firewall instead of having a direct interface in the storage VLAN.

I am currently doing some final testing, but NFS 4.1 is now behaving normally and the previous long stalls are gone.

Thanks again for pointing me towards cleaning up the old storage definitions and mounts — that gave me a clean baseline for troubleshooting.
 
  • Like
Reactions: driley