Invisible ZFS dataset on zfs 2.2.2 kernel 6.5.11-7-pve

[aabbccdd]

Directory '/rust/media/public' is not visible anymore.

root@server:~# ls -la /rust/media/
total 27
drwxr-xr-x  5 root root  5 Jun 23 06:24 .
drwxr-xr-x  8 root root  8 Dec  2 01:29 ..
drwxrwx---  8 plex plex 10 Dec  4 05:39 config
drwxrwx---+ 9 plex plex 28 Nov 27 01:15 private
drwx------  2 root root  2 Dec  8 19:50 root

But /rust/media/public does exist!

root@server:~# zfs list | grep public
rust/media/public                               12.4G  13.4T  12.4G  /rust/media/public
root@server:~# ls -la /rust/media/public
total 43
drwxrwxr-x 5 mc       mc   5 Dec  2 07:16 .
drwxr-xr-x 5 root     root 5 Jun 23 06:24 ..
drwxrwxr-x 7 mc       mc   7 Dec  2 07:39 done
drwxrwxr-x 7 usenet   mc   7 Dec  7 21:20 usenet

It seems to be mounted.
root@server:~# zfs list -o name,mounted | egrep 'NAME|rust/media/public' NAME MOUNTED rust/media yes rust/media/public yes


When moving the dir named 'root' out of the way, it breaks.

root@server:~# mv /rust/media/root/ /tmp
root@server:~# ls -l /rust/media/public
ls: cannot access '/rust/media/public': Input/output error

But can not be mounted 'file exists'?!

root@server:~# zfs mount rust/media/public
cannot mount '/rust/media/public': failed to create mountpoint: File exists

I'm using the latest software versions for everything and I'm running kernel 6.5.11-7. Seems to be a ZFS related issue. Using the latest proxmox and zfs v2.2.2.

Clues on how to troubleshoot or fix this odd behavior?

Update on 18DEC23: Posted the question on the OpenZFS mailinglist since it might be more relevant to ZFS than to proxmox.

 

[LordRatner]

What does the full output of zfs list show?

 

[aabbccdd]

Output of zfs list

root@server:~# zfs list | egrep 'NAME|rust/media'
NAME                                             USED  AVAIL  REFER  MOUNTPOINT
rust/media                                       810G  13.2T   112K  /rust/media
rust/media/config                                210G  13.2T   203G  /rust/media/config
rust/media/private                               588G  13.2T   535G  /rust/media/private
rust/media/public                               12.4G  13.2T  12.4G  /rust/media/public

 

[LnxBil]

have you tried mounting it? ZFS datasets are only mounted if the mountpoint is empty. This is the default and can be changed.

You can view the mount status with

$ zfs list -o name,mounted | egrep 'NAME|rust/media'

 

[aabbccdd]

Not sure if I get where you are going with the mount status.

On reboot, the dataset is mounted until the /rust/media/root directory is removed, than it breaks (can't be mounted or unmounted anymore as shown in my opening message).

Currently it is mounted
root@server:~# zfs list -o name,mounted | egrep 'NAME|rust/media' NAME MOUNTED rust/media yes rust/media/config yes rust/media/private yes rust/media/public yes

Do you recognize the fact that a mountpoint is replaced with the directory named root? Or, how can I troubleshoot / understand better what is happening here.

The 'root' dir seems empty
root@server:~# ls -l /rust/media/ total 18 drwxrwx--- 8 plex plex 10 Dec 4 05:39 config drwxrwx---+ 9 plex plex 81 Dec 15 19:15 private drwx------ 2 root root 2 Dec 17 20:54 root root@server:~# ls -la /rust/media/root/ total 1 drwx------ 2 root root 2 Dec 18 07:04 . drwxr-xr-x 5 root root 5 Jun 23 06:24 ..

 

[aabbccdd]

A clue anyone?