[SOLVED] One of my nodes can't access NFS share

[molnart]

So, this is a strange issue, as I think shared storage is configured on datacenter level. Yet, on of my nodes can't access the share.

I can manually mount the share with nfs vers=3 only, with vers=4 it gives an error.

both nodes are on the same version. node pve-smc was restarted recently. I am clueless where the problem lies.

root@pve-smc:/# pvesm nfsscan 192.168.50.8
/export         192.168.0.0/16,192.168.110.0/24,192.168.50.0/24
/export/Backups 192.168.50.0/24
/export/Images  192.168.0.0/16
root@pve-smc:/# mount -t nfs -o vers=4 192.168.50.8:/export/Backups /mnt/WD60
mount.nfs: mounting 192.168.50.8:/export/Backups failed, reason given by server: No such file or directory
root@pve-smc:/# mount -t nfs -o vers=3 192.168.50.8:/export/Backups /mnt/WD60

 

[Chris]

reason given by server: No such file or directory

Hi,
looks more link an issue on the server side. Do you get any hints in the journal of the NFS server what might be off?

 

[molnart]

yes, it looks like that, and it was also my first hunch, but after seeing that my another PVE node is able to connect to it, i am leaning towards PVE being the culprit.

etc/pve/storage.cfg looks the same on both nodes

root@pve-smc:~# cat /etc/pve/storage.cfg
dir: local
        path /var/lib/vz
        content vztmpl,iso,backup

lvmthin: local-lvm
        thinpool data
        vgname pve
        content rootdir,images

nfs: backup-omv
        export /export/Backups
        path /mnt/pve/backup-omv
        server 192.168.50.8
        content backup,images
        prune-backups keep-last=5

lvmthin: local-lvm-ext
        thinpool local-lvm-ext
        vgname local-lvm-ext
        content rootdir,images
        nodes pve-smc
       
root@pve-topton:~# cat /etc/pve/storage.cfg
dir: local
        path /var/lib/vz
        content vztmpl,iso,backup

lvmthin: local-lvm
        thinpool data
        vgname pve
        content rootdir,images

nfs: backup-omv
        export /export/Backups
        path /mnt/pve/backup-omv
        server 192.168.50.8
        content backup,images
        prune-backups keep-last=5

lvmthin: local-lvm-ext
        thinpool local-lvm-ext
        vgname local-lvm-ext
        content rootdir,images
        nodes pve-smc

EDIT: strangely, solved by umount /mnt/pve/backup-omv and the share just appeared

 

[Chris]

yes, it looks like that, and it was also my first hunch, but after seeing that my another PVE node is able to connect to it, i am leaning towards PVE being the culprit.

etc/pve/storage.cfg looks the same on both nodes

root@pve-smc:~# cat /etc/pve/storage.cfg
dir: local
        path /var/lib/vz
        content vztmpl,iso,backup

lvmthin: local-lvm
        thinpool data
        vgname pve
        content rootdir,images

nfs: backup-omv
        export /export/Backups
        path /mnt/pve/backup-omv
        server 192.168.50.8
        content backup,images
        prune-backups keep-last=5

lvmthin: local-lvm-ext
        thinpool local-lvm-ext
        vgname local-lvm-ext
        content rootdir,images
        nodes pve-smc
     
root@pve-topton:~# cat /etc/pve/storage.cfg
dir: local
        path /var/lib/vz
        content vztmpl,iso,backup

lvmthin: local-lvm
        thinpool data
        vgname pve
        content rootdir,images

nfs: backup-omv
        export /export/Backups
        path /mnt/pve/backup-omv
        server 192.168.50.8
        content backup,images
        prune-backups keep-last=5

lvmthin: local-lvm-ext
        thinpool local-lvm-ext
        vgname local-lvm-ext
        content rootdir,images
        nodes pve-smc

EDIT: strangely, solved by umount /mnt/pve/backup-omv and the share just appeared

So maybe an existing mount interfering with the NFS storage plugin. As you noted, all contents located under /etc/pve are the same on all cluster nodes. This is by design, as they are shared via the proxmox cluster filesystem, see https://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_pmxcfs.

So all nodes which are part of the quorate cluster partition must have the same contents, as they share the same state.