I have a server with 5 nvme disks, 1 for the OS and 4 for data.
And because of random assigning at boot of the numbers, this sometimes results in one disk missing from the zpool. So I did a bit of digging and found out it's better to assign the serial number to the array member. And I wrote a script around it, which prints the right commands to replace the zpool member names.
So for every disk you have to do this once. The disk needs to be resilvered. Per disk, it took about half an hour.
Of course, this is not entirely without risk, make sure you have backups, don't run this on production machines etc.
I recommend you run the script as is — it doesn't do anything but print the right commands — double-check the output and then C&P it.
I hope the proxmox devs will change the installer so that this layout will be standard for zpools. It's a minor change in the installer and will save a major headache for setups that might be affected.
Code:
# lsblk /dev/nvme{0..4}n1
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
nvme0n1 259:0 0 894.3G 0 disk
├─nvme0n1p1 259:1 0 894.2G 0 part
└─nvme0n1p9 259:2 0 8M 0 part
nvme1n1 259:3 0 894.3G 0 disk
├─nvme1n1p1 259:19 0 894.2G 0 part
└─nvme1n1p9 259:20 0 8M 0 part
nvme3n1 259:4 0 894.3G 0 disk
├─nvme3n1p1 259:17 0 894.2G 0 part
└─nvme3n1p9 259:18 0 8M 0 part
nvme2n1 259:5 0 894.3G 0 disk
├─nvme2n1p1 259:8 0 894.2G 0 part
└─nvme2n1p9 259:9 0 8M 0 part
nvme4n1 259:13 0 447.1G 0 disk
├─nvme4n1p1 259:14 0 1007K 0 part
├─nvme4n1p2 259:15 0 1G 0 part
└─nvme4n1p3 259:16 0 446.1G 0 part /And because of random assigning at boot of the numbers, this sometimes results in one disk missing from the zpool. So I did a bit of digging and found out it's better to assign the serial number to the array member. And I wrote a script around it, which prints the right commands to replace the zpool member names.
Bash:
#!/bin/sh
dev=$(zpool status rpool | awk '/nvme.n1.*ONLINE/{print $1; exit}')
id=$(lsblk -ndo ID-LINK /dev/$dev)
echo zpool offline rpool $dev
echo zpool labelclear -f /dev/${dev}p1
echo wipefs -a /dev/$dev
echo zpool replace rpool $dev /dev/disk/by-id/$id
echo watch zpool statusSo for every disk you have to do this once. The disk needs to be resilvered. Per disk, it took about half an hour.
Of course, this is not entirely without risk, make sure you have backups, don't run this on production machines etc.
I recommend you run the script as is — it doesn't do anything but print the right commands — double-check the output and then C&P it.
I hope the proxmox devs will change the installer so that this layout will be standard for zpools. It's a minor change in the installer and will save a major headache for setups that might be affected.
Last edited: