[SOLVED] sharenfs stopped working with latest non subscription repo

[Ivan Dimitrov]

I have a few zfs datasets shared with VM and between proxmox instances exported by proxmox with sharenfs. After the latest upgrade the shares are not exported anymore. In the journal I don't see any particular error besides the host trying to mount missing export.
Also one strange thing during the upgrade was that I was asked if I want to keep some of the zfs configuration files as they seems to have changed. I didn't pay much attention to what were those files and what changed but I guess the issue is there.

After I run zfs share -a the exports are shown for a while and disappear again.

Any clues how to debug further?

proxmox-ve: 8.0.2 (running kernel: 6.2.16-19-pve)
pve-manager: 8.0.9 (running version: 8.0.9/fd1a0ae1b385cdcd)
pve-kernel-6.2: 8.0.5
proxmox-kernel-helper: 8.0.5
pve-kernel-5.15: 7.4-4
proxmox-kernel-6.2.16-19-pve: 6.2.16-19
proxmox-kernel-6.2: 6.2.16-19
proxmox-kernel-6.2.16-15-pve: 6.2.16-15
pve-kernel-5.15.108-1-pve: 5.15.108-1
ceph-fuse: 16.2.11+ds-2
corosync: 3.1.7-pve3
criu: 3.17.1-2
glusterfs-client: 10.3-5
ifupdown: not correctly installed
ifupdown2: 3.2.0-1+pmx6
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-4
libknet1: 1.28-pve1
libproxmox-acme-perl: 1.5.0
libproxmox-backup-qemu0: 1.4.0
libproxmox-rs-perl: 0.3.1
libpve-access-control: 8.0.6
libpve-apiclient-perl: 3.3.1
libpve-common-perl: 8.0.10
libpve-guest-common-perl: 5.0.5
libpve-http-server-perl: 5.0.5
libpve-rs-perl: 0.8.7
libpve-storage-perl: 8.0.4
libqb0: 1.0.5-1
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 5.0.2-4
lxcfs: 5.0.3-pve3
novnc-pve: 1.4.0-2
openvswitch-switch: 3.1.0-2
proxmox-backup-client: 3.0.4-1
proxmox-backup-file-restore: 3.0.4-1
proxmox-kernel-helper: 8.0.5
proxmox-mail-forward: 0.2.1
proxmox-mini-journalreader: 1.4.0
proxmox-offline-mirror-helper: 0.6.2
proxmox-widget-toolkit: 4.1.1
pve-cluster: 8.0.5
pve-container: 5.0.5
pve-docs: 8.0.5
pve-edk2-firmware: 3.20230228-4
pve-firewall: 5.0.3
pve-firmware: 3.9-1
pve-ha-manager: 4.0.3
pve-i18n: 3.0.7
pve-qemu-kvm: 8.1.2-2
pve-xtermjs: 5.3.0-2
qemu-server: 8.0.8
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.2.0-pve3

 

[Ivan Dimitrov]

After some debugging the file /etc/exports.d/zfs.exports is empty and being modified constantly every minute or so.

Also found some changes in zfs 2.2 related to the nfs export:

• https://github.com/openzfs/zfs/commit/a3d387b31de44e2069c00bed18491e032c7424dc
• https://github.com/openzfs/zfs/commit/8357bbff1fdf99554d6a296dee4841e41d87b7c9
• https://github.com/openzfs/zfs/commit/dc3a56d38bf300cf15f13d5bd4b116ec387c60e0
• https://github.com/openzfs/zfs/commit/62866fc96cab95516cd9625839da6a75599b7931

EDIT:

I downgraded the following packages and the exports started working again. There is definitely a bug in zfs 2.2

zfs-zed:=2.1.13-pve1 zfs-initramfs:=2.1.13-pve1 spl:=2.1.13-pve1 libnvpair3linux:=2.1.13-pve1 libuutil3linux:=2.1.13-pve1 libzpool5linux:=2.1.13-pve1 libzfs4linux:=2.1.13-pve1 zfsutils-linux:=2.1.13-pve1

 

[Mark Verboom]

Thanks for the workaround! Did some more testing myself. When I re set the dataset property for sharenfs without downgrading the packages, the nfs shares also come back.
Did this oneliner for my data share:

zfs get sharenfs -o name,value -H -r -s local data | while read name value; do zfs set sharenfs="$value" "$name"; done

Unfortunatly, this doesn't seem to keep after a reboot. Requires some more investigation.

Regards,

Mark

 

[Ivan Dimitrov]

Thanks for the workaround! Did some more testing myself. When I re set the dataset property for sharenfs without downgrading the packages, the nfs shares also come back.
Did this oneliner for my data share:

zfs get sharenfs -o name,value -H -r -s local data | while read name value; do zfs set sharenfs="$value" "$name"; done

Unfortunatly, this doesn't seem to keep after a reboot. Requires some more investigation.

Regards,

Mark

Unfortunately your workaround didn't work for me. After executing the oneliner nothing changes.
I opened bug report I hope this get the attention this issue deserves.
I know zfs 2.2 has other more significant issues but this one seems like a significant regression too.

 

[Mark Verboom]

Unfortunately your workaround didn't work for me. After executing the oneliner nothing changes.
I opened bug report I hope this get the attention this issue deserves.
I know zfs 2.2 has other more significant issues but this one seems like a significant regression too.

That is weird? Did you change the name of the pool in the oneliner (data)?
Did run this om my server and before when doing an showmount -e was empty and afterwards it shows the exported filesystems..

Regards,

Mark

 

[Ivan Dimitrov]

I ran the command with multiple pools at once. Not sure if this had an impact.

 

[Ivan Dimitrov]

The zfs 2.2.2-pve-2 fixed the issue. Thank you for the help.