critical medium error in nvme zfs pool, no boot

PaulHurleyuk

New Member
Feb 3, 2025
2
0
1
I built my own home server about two years ago, which has a pair of wd_black nvme drives as the boot volume in a zfs pool. This has been running well since then but today it shutdown and won't reboot, I get the following messages (retyped from screen)
[650.4249121] nvme0n1: 1/0 Cmd(0x1) @ LBA 1051152, 16 blocks, 1/0 Error (sct 0x1 / sc 0x82) DNR
[650.425249] critical medium error, dev nvme0n1 sector 1051152 op 0x1: (WRITE) flags 0x0 phys_seg 1 prio class 0
[650.4255921] zio pool=rpool udev=/dev/disk/by-id/nvme-eui.e8238fa6bf530001001b444a488cf34c-part3 error=61 type=2 offset=2703
size=8192 flags=721089
[650.4255921] zio pool=rpool udev=/dev/disk/by-id/nvme-eui.e8238fa6bf530001001b444a488cf34c-part3 error=61 type=2 offset=9996
229248 size=8192 flags=721089
[650.4255921] zio pool=rpool udev=/dev/disk/by-id/nvme-eui.e8238fa6bf530001001b444a488cf34c-part3 error=61 type=2 offset=9996
491392 size=8192 flags=721089
cannot import 'rpool': one or more devices is currently unavailable

I don't know much about nvme drive support or zfs, but I tried running zpool import and zpool status and get;

zpool import
pool: rpool
id: 18292245705162545634
state: online
status: some supported features are not enabled on the pooll.
<snip>
config:
rpool
nvme-eui.e8238fa6..............c3000-part3 ONLINE
nvme-eui.e8238fa6.............cf34c-part3 ONLINE


Is there any way to recover and boot the system ?
 
Two years with those cheap SSDs as a Proxmox ZFS root pool is pretty good!
But as Walter already said, it looks like a stripe set not a mirror set.
After "zpool import" it said "state: online"
Try
Code:
zpool status
and
Code:
df -h
to see if you still can access.
 
One of the best features of ZFS is redundancy... which requires actually redundant drives.

If you still can not import it: read man zpool-import. There are some options to import a damaged pool. I have no experience with this, but "-F -n" looks interesting.
 
thanks both, I thought it was a mirror but does look like I was wrong.

Code:
>zpool status
no pools available

Code:
>df -h
shows udev mounted on /dev, and tmpfs mounted on /run

so looks like I need to wipe and rebuild. This box has a part of disks which I think are mirror'd, they show up as

Code:
pool: storage

config:
    storage                 ONLINE
        mirror-0            ONLINE
            ata-wdc.....   ONLINE
            ata-wdc.....   ONLINE

Can someone point me to a guide or docs on how to recover that storage after I rebuild (or boot from a usb)
 
Did you see #4 and tried "zpool import -F -n"?