how to mount nfs auto

R

restartxie

New Member
Jan 13, 2023
5
0
1
Dear ALL,when i do some test ,i have some problems,if i reboot all the node in the cluster,after node rebooed,the nfs lun cannot mount automatical,the status is unknow,i must modify the nfs properties or remount ,so how can i solve it.thanks.
pve version: 8.14
storage: netapp
 
restartxie said:
he nfs lun cannot mount automatical,the status is unknow
Click to expand...
Have you checked the logs (journalctl -b0) and are there any errors that you think might be related to nfs?
restartxie said:
,i must modify the nfs properties
Click to expand...
which properties? where? and how do you modify them?
restartxie said:
remount
Click to expand...
remount how?
restartxie said:
so how can i solve it.thanks.
pve version: 8.14
storage: netapp
Click to expand...
What you are describing is not the expected behavior. Considering that many people use PVE&NFS without any issues, this situation is specific to your setup. You need to provide more details for forum members to be able to help:

cat /etc/pve/storage.cfg
pvesm status (in good state)
pvesm status (in bad state)
journalctl -f (for a few minutes during bad state)
describe how you are able to fix it in details

good luck

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
thanks for you replay,this is the info .if i reboot one of the node ,it's ok.but if i reboot all the nodes at the same time ,it doesn't work.
1:
root@pve01:~# journalctl -b0 | grep nfs
Apr 11 13:52:12 pve01 systemd[1]: Reached target nfs-client.target - NFS client services.
Apr 11 13:52:42 pve01 pvestatd[1282]: mount error: mount.nfs: Operation not permitted
Apr 11 13:52:42 pve01 pvestatd[1282]: mount error: mount.nfs: Operation not permitted
Apr 11 13:52:46 pve01 pvestatd[1282]: mount error: mount.nfs: Operation not permitted
Apr 11 13:52:46 pve01 pvestatd[1282]: mount error: mount.nfs: Operation not permitted
Apr 11 13:52:54 pve01 pvestatd[1282]: mount error: mount.nfs: Operation not permitted
Apr 11 13:52:57 pve01 pvestatd[1282]: mount error: mount.nfs: Operation not permitted
Apr 11 13:52:59 pve01 pvedaemon[1310]: mount error: mount.nfs: Operation not permitted
Apr 11 13:53:01 pve01 pvedaemon[1309]: mount error: mount.nfs: Operation not permitted
Apr 11 13:53:01 pve01 pvedaemon[1309]: mount error: mount.nfs: Operation not permitted
Apr 11 13:53:02 pve01 pvedaemon[1310]: mount error: mount.nfs: Operation not permitted
Apr 11 13:53:03 pve01 pvestatd[1282]: mount error: mount.nfs: Operation not permitted
Apr 11 13:53:04 pve01 pvedaemon[1310]: mount error: mount.nfs: Operation not permitted
Apr 11 13:53:04 pve01 pvestatd[1282]: mount error: mount.nfs: Operation not permitted
Apr 11 13:53:08 pve01 pvedaemon[1310]: mount error: mount.nfs: Operation not permitted
Apr 11 13:53:14 pve01 pvestatd[1282]: mount error: mount.nfs: Operation not permitted
Apr 11 13:53:14 pve01 pvestatd[1282]: mount error: mount.nfs: Operation not permitted
Apr 11 13:53:23 pve01 pvestatd[1282]: mount error: mount.nfs: Operation not permitted
Apr 11 13:53:24 pve01 pvestatd[1282]: mount error: mount.nfs: Operation not permitted
Apr 11 13:53:33 pve01 pvestatd[1282]: mount error: mount.nfs: Operation not permitted
Apr 11 13:53:36 pve01 pvestatd[1282]: mount error: mount.nfs: Operation not permitted
Apr 11 13:53:44 pve01 pvestatd[1282]: mount error: mount.nfs: Operation not permitted
Apr 11 13:53:47 pve01 pvestatd[1282]: mount error: mount.nfs: Operation not permitted
Apr 11 13:53:53 pve01 pvestatd[1282]: mount error: mount.nfs: Operation not permitted
Apr 11 13:53:54 pve01 pvestatd[1282]: mount error: mount.nfs: Operation not permitted
Apr 11 13:54:03 pve01 pvestatd[1282]: mount error: mount.nfs: Operation not permitted
Apr 11 13:54:07 pve01 pvestatd[1282]: mount error: mount.nfs: Operation not permitted
Apr 11 13:54:13 pve01 pvestatd[1282]: mount error: mount.nfs: Operation not permitted
Apr 11 13:54:16 pve01 pvestatd[1282]: mount error: mount.nfs: Operation not permitted
Apr 11 13:54:22 pve01 pve-ha-lrm[2200]: mount error: mount.nfs: Operation not permitted
Apr 11 13:54:22 pve01 pve-ha-lrm[2199]: <root@pam> end task UPID:pve01:00000898:00003695:66177B0E:qmstart:105:root@pam: mount error: mount.nfs: Operation not permitted
Apr 11 13:54:22 pve01 pve-ha-lrm[2201]: mount error: mount.nfs: Operation not permitted
Apr 11 13:54:22 pve01 pve-ha-lrm[2198]: <root@pam> end task UPID:pve01:00000899:00003695:66177B0E:qmstart:102:root@pam: mount error: mount.nfs: Operation not permitted
Apr 11 13:54:24 pve01 pvestatd[1282]: mount error: mount.nfs: Operation not permitted
Apr 11 13:54:27 pve01 pvestatd[1282]: mount error: mount.nfs: Operation not permitted
Apr 11 13:54:32 pve01 pve-ha-lrm[2284]: mount error: mount.nfs: Operation not permitted
Apr 11 13:54:32 pve01 pve-ha-lrm[2282]: <root@pam> end task UPID:pve01:000008EC:00003A96:66177B18:qmstart:102:root@pam: mount error: mount.nfs: Operation not permitted
Apr 11 13:54:32 pve01 pve-ha-lrm[2285]: mount error: mount.nfs: Operation not permitted
Apr 11 13:54:32 pve01 pve-ha-lrm[2283]: <root@pam> end task UPID:pve01:000008ED:00003A96:66177B18:qmstart:105:root@pam: mount error: mount.nfs: Operation not permitted


2: i have test two methods
first: unmont the nfs then mount it again ,it can work again
or edit the nfs version from default to four ,it can work agin

3:
nfs: NFS_LUN01
export /vol_NFS_PVE_01
path /mnt/pve/NFS_LUN01
server 10.1.1.16
content images
options vers=4
prune-backups keep-all=1

nfs: NFS_LUN02
export /vol_NFS_PVE_02
path /mnt/pve/NFS_LUN02
server 10.1.1.16
content images
options vers=4
prune-backups keep-all=1

root@pve01:~# pvesm status (in good status)
Name Type Status Total Used Available %
PVE_TEST01 iscsi active 0 0 0 0.00%
ISCSI_LUN_01 lvm active 104853504 83886080 20967424 80.00%
NFS_LUN01 nfs active 498073600 32095040 465978560 6.44%
NFS_LUN02 nfs active 498073600 172309056 325764544 34.60%
iso cifs active 9961472 9044008 917464 90.79%
local dir active 44867864 4066620 38489668 9.06%
local-lvm lvmthin active 68513792 0 68513792 0.00%
pbssrv pbs active 308520748 20080448 272695332 6.51%

root@pve01:~# pvesm status(in bad status )
mount error: mount.nfs: Operation not permitted
pbssrv: error fetching datastores - 500 Can't connect to 10.1.7.231:8007 (No route to host)
mount error: mount.nfs: Operation not permitted
Name Type Status Total Used Available %
PVE_TEST01 iscsi active 0 0 0 0.00%
ISCSI_LUN_01 lvm active 104853504 83886080 20967424 80.00%
NFS_LUN01 nfs inactive 0 0 0 0.00%
NFS_LUN02 nfs inactive 0 0 0 0.00%
iso cifs active 9961472 9044008 917464 90.79%
local dir active 44867864 4074152 38482136 9.08%
local-lvm lvmthin active 68513792 0 68513792 0.00%
pbssrv pbs inactive 0 0 0 0.00%


root@pve01:~# journalctl -f
Apr 12 09:03:14 pve01 pvestatd[1291]: pbssrv: error fetching datastores - 500 Can't connect to 10.1.1.231:8007 (No route to host)
Apr 12 09:03:14 pve01 pvestatd[1291]: mount error: mount.nfs: Operation not permitted
Apr 12 09:03:14 pve01 pvestatd[1291]: mount error: mount.nfs: Operation not permitted
Apr 12 09:03:22 pve01 pvestatd[1291]: mount error: mount.nfs: Operation not permitted
Apr 12 09:03:22 pve01 kernel: sd 7:0:0:5: alua: supports implicit TPGS
Apr 12 09:03:22 pve01 kernel: sd 7:0:0:5: alua: device naa.600a098038302d50613f49396b75306a port group 3e8 rel port 3
Apr 12 09:03:22 pve01 kernel: sd 8:0:0:5: alua: supports implicit TPGS
Apr 12 09:03:22 pve01 kernel: sd 8:0:0:5: alua: device naa.600a098038302d50613f49396b75306a port group 3e9 rel port 4
Apr 12 09:03:22 pve01 pvestatd[1291]: mount error: mount.nfs: Operation not permitted
Apr 12 09:03:25 pve01 pvestatd[1291]: pbssrv: error fetching datastores - 500 Can't connect to 10.1.1.231:8007 (No route to host)
Apr 12 09:03:35 pve01 pvestatd[1291]: pbssrv: error fetching datastores - 500 Can't connect to 10.1.1.231:8007 (No route to host)
Apr 12 09:03:35 pve01 kernel: sd 7:0:0:5: alua: supports implicit TPGS
Apr 12 09:03:35 pve01 kernel: sd 7:0:0:5: alua: device naa.600a098038302d50613f49396b75306a port group 3e8 rel port 3
Apr 12 09:03:35 pve01 kernel: sd 8:0:0:5: alua: supports implicit TPGS
Apr 12 09:03:35 pve01 kernel: sd 8:0:0:5: alua: device naa.600a098038302d50613f49396b75306a port group 3e9 rel port 4
Apr 12 09:03:35 pve01 pvestatd[1291]: mount error: mount.nfs: Operation not permitted
Apr 12 09:03:35 pve01 pvestatd[1291]: mount error: mount.nfs: Operation not permitted
Apr 12 09:03:41 pve01 pvestatd[1291]: mount error: mount.nfs: Operation not permitted
Apr 12 09:03:41 pve01 pvestatd[1291]: mount error: mount.nfs: Operation not permitted
Apr 12 09:03:45 pve01 pvestatd[1291]: pbssrv: error fetching datastores - 500 Can't connect to 10.1.1.231:8007 (No route to host)
Apr 12 09:03:51 pve01 pvestatd[1291]: mount error: mount.nfs: Operation not permitted
Apr 12 09:03:51 pve01 kernel: sd 7:0:0:5: alua: supports implicit TPGS
Apr 12 09:03:51 pve01 kernel: sd 7:0:0:5: alua: device naa.600a098038302d50613f49396b75306a port group 3e8 rel port 3
 
bbgeek17 said:
Have you checked the logs (journalctl -b0) and are there any errors that you think might be related to nfs?

which properties? where? and how do you modify them?

remount how?

What you are describing is not the expected behavior. Considering that many people use PVE&NFS without any issues, this situation is specific to your setup. You need to provide more details for forum members to be able to help:

cat /etc/pve/storage.cfg
pvesm status (in good state)
pvesm status (in bad state)
journalctl -f (for a few minutes during bad state)
describe how you are able to fix it in details

good luck

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
Click to expand...
thanks for your reply,
 
restartxie said:
or edit the nfs version from default to four ,it can work agin
Click to expand...
What is the output of "rpcinfo 10.1.1.16" and "showmount 10.1.1.16"?
If "vers 4" works, why not leave it in the config?

What is the output of "mount" in good state, bad state and fixed state?

Proxmox uses Linux NFS to mount your NAS, the error message is coming from NAS.

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
bbgeek17 said:
What is the output of "rpcinfo 10.1.1.16" and "showmount 10.1.1.16"?
Click to expand...
root@pve01:~# rpcinfo 10.1.1.16
program version netid address service owner
100000 2 udp 10.1.1.16.0.111 portmapper
100000 2 tcp 10.1.1.16.0.111 portmapper
100000 3 udp 10.1.1.16.0.111 portmapper
100000 3 tcp 10.1.1.16.0.111 portmapper
100000 4 udp 10.1.1.16.0.111 portmapper
100000 4 tcp 10.1.1.16.0.111 portmapper
100003 3 udp 10.1.1.16.8.1 nfs
100003 3 tcp 10.1.1.16.8.1 nfs
100003 4 tcp 10.1.1.16.8.1 nfs
400010 1 tcp 10.1.1.16.8.1 -
100005 1 udp 10.1.1.16.2.123 mountd
100005 2 udp 10.1.1.16.2.123 mountd
100005 3 udp 10.1.1.16.2.123 mountd
100005 1 tcp 10.1.1.16.2.123 mountd
100005 2 tcp 10.1.1.16.2.123 mountd
100005 3 tcp 10.1.1.16.2.123 mountd
100021 4 udp 10.1.1.16.15.205 nlockmgr
100021 4 tcp 10.1.1.16.15.205 nlockmgr
100024 1 udp 10.1.1.16.15.206 status
100024 1 tcp 10.1.1.16.15.206 status
100011 1 udp 10.1.1.16.15.209 rquotad
root@pve01:~# showmount 10.1.1.16
Hosts on 10.1.1.16:
bbgeek17 said:
If "vers 4" works, why not leave it in the config?
Click to expand...
but if i reboot all the nodes at the same time,it will not work again.i will edit it from verion 4 to default ,it will work again
bbgeek17 said:
What is the output of "mount" in good state, bad state and fixed state?
Click to expand...
root@pve01:~# mount (in bad state)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,relatime)
udev on /dev type devtmpfs (rw,nosuid,relatime,size=32908668k,nr_inodes=8227167,mode=755,inode64)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,noexec,relatime,size=6588488k,mode=755,inode64)
/dev/mapper/pve-root on / type ext4 (rw,relatime,errors=remount-ro)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,inode64)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k,inode64)
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=31363)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime)
tracefs on /sys/kernel/tracing type tracefs (rw,nosuid,nodev,noexec,relatime)
fusectl on /sys/fs/fuse/connections type fusectl (rw,nosuid,nodev,noexec,relatime)
configfs on /sys/kernel/config type configfs (rw,nosuid,nodev,noexec,relatime)
ramfs on /run/credentials/systemd-sysusers.service type ramfs (ro,nosuid,nodev,noexec,relatime,mode=700)
ramfs on /run/credentials/systemd-tmpfiles-setup-dev.service type ramfs (ro,nosuid,nodev,noexec,relatime,mode=700)
ramfs on /run/credentials/systemd-sysctl.service type ramfs (ro,nosuid,nodev,noexec,relatime,mode=700)
ramfs on /run/credentials/systemd-tmpfiles-setup.service type ramfs (ro,nosuid,nodev,noexec,relatime,mode=700)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,nosuid,nodev,noexec,relatime)
sunrpc on /run/rpc_pipefs type rpc_pipefs (rw,relatime)
lxcfs on /var/lib/lxcfs type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
/dev/fuse on /etc/pve type fuse (rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other)

root@pve01:~# mount (in good state)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,relatime)
udev on /dev type devtmpfs (rw,nosuid,relatime,size=32908668k,nr_inodes=8227167,mode=755,inode64)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,noexec,relatime,size=6588488k,mode=755,inode64)
/dev/mapper/pve-root on / type ext4 (rw,relatime,errors=remount-ro)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,inode64)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k,inode64)
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=31363)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime)
tracefs on /sys/kernel/tracing type tracefs (rw,nosuid,nodev,noexec,relatime)
fusectl on /sys/fs/fuse/connections type fusectl (rw,nosuid,nodev,noexec,relatime)
configfs on /sys/kernel/config type configfs (rw,nosuid,nodev,noexec,relatime)
ramfs on /run/credentials/systemd-sysusers.service type ramfs (ro,nosuid,nodev,noexec,relatime,mode=700)
ramfs on /run/credentials/systemd-tmpfiles-setup-dev.service type ramfs (ro,nosuid,nodev,noexec,relatime,mode=700)
ramfs on /run/credentials/systemd-sysctl.service type ramfs (ro,nosuid,nodev,noexec,relatime,mode=700)
ramfs on /run/credentials/systemd-tmpfiles-setup.service type ramfs (ro,nosuid,nodev,noexec,relatime,mode=700)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,nosuid,nodev,noexec,relatime)
sunrpc on /run/rpc_pipefs type rpc_pipefs (rw,relatime)
lxcfs on /var/lib/lxcfs type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
/dev/fuse on /etc/pve type fuse (rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=6588484k,nr_inodes=1647121,mode=700,inode64)
10.1.1.16:/vol_NFS_PVE_01 on /mnt/pve/NFS_LUN01 type nfs (rw,relatime,vers=3,rsize=65536,wsize=65536,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.1.1.16,mountvers=3,mountport=635,mountproto=udp,local_lock=none,addr=10.1.1.16)
10.1.1.16:/vol_NFS_PVE_02 on /mnt/pve/NFS_LUN02 type nfs (rw,relatime,vers=3,rsize=65536,wsize=65536,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.1.1.16,mountvers=3,mountport=635,mountproto=udp,local_lock=none,addr=10.1.1.16)


bbgeek17 said:
Proxmox uses Linux NFS to mount your NAS, the error message is coming from NAS.
Click to expand...
wo,but if the problem happen,i only modify the nfs version to others ,didn't modify the storage ,it will solved.or i think if i change any properties of the nfs, it will work also

bbgeek17 said:
Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
Click to expand...
 
restartxie said:
Dear ALL,when i do some test ,i have some problems,if i reboot all the node in the cluster,after node rebooed,the nfs lun cannot mount automatical,the status is unknow,i must modify the nfs properties or remount ,so how can i solve it.thanks.
pve version: 8.14
storage: netapp
Click to expand...
Just wondering - where is the NFS located? I'm hoping its not a VM on one of the nodes.
 
restartxie said:
but if i reboot all the nodes at the same time
Click to expand...
I am not sure, hopefully there is more in the log (check from node start). Its very strange and no one has reported a similar issue before, although I dont know how many people regularly reboot all nodes in their cluster. I would definitely study the log very carefully.

If you have free time and desire, try to get it in bad state, then restart PVE services on one node at a time - note if that remounts things.

Essentially, you are saying that PVE is unable to mount NFS on fresh boot. On an off chance, check permissions on local directory in /mnt/pve


Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back