Problem:
I've tried installing PVE on a system based on an Asrock Rack x470-D4U motherboard with a pair of NVMe drives using ZFS RAID1 as the root filesystem. I've used 6.3-1, 6.2-1, and 6.1-1 ISOs with identical results. The system is configured for UEFI boot, not legacy boot.
At the end of the installation process, the installer gives this error:
The console has this error, for both nvme0 and nvme1 when running pve-efiboot-tool:
Further investigation shows that p2 is set up as an mdadm mirror with v1.0 metadata. mdadm itself isn't installed. After installing mdadm and bringing md0 online, md0 mounts, but is empty. Furthermore, running pve-efiboot-tool against md0 gives an error saying it wants a partition, not a raw device.
However, on the same system, if I ignore the NVMe drives and install PVE to a pair of SATA hard drives, with ZFS RAID1, the install completes without errors. The p2 ESP partitions are not setup as mdadm devices, and do actually contain a vfat filesystem with the appropriate files one would expect in the ESP.
The above behavior seems like a bug in the installer.
Workaround:
Since I had the system booted successfully from the SATA drives, I did the following:
Thanks.
I've tried installing PVE on a system based on an Asrock Rack x470-D4U motherboard with a pair of NVMe drives using ZFS RAID1 as the root filesystem. I've used 6.3-1, 6.2-1, and 6.1-1 ISOs with identical results. The system is configured for UEFI boot, not legacy boot.
At the end of the installation process, the installer gives this error:
Code:
unable to init ESP and install systemd-boot loader on '/dev/nvme0n1p2'The console has this error, for both nvme0 and nvme1 when running pve-efiboot-tool:
Code:
E: '/dev/nvme0n1p2' has wrong filesystem (!= vfat)Further investigation shows that p2 is set up as an mdadm mirror with v1.0 metadata. mdadm itself isn't installed. After installing mdadm and bringing md0 online, md0 mounts, but is empty. Furthermore, running pve-efiboot-tool against md0 gives an error saying it wants a partition, not a raw device.
However, on the same system, if I ignore the NVMe drives and install PVE to a pair of SATA hard drives, with ZFS RAID1, the install completes without errors. The p2 ESP partitions are not setup as mdadm devices, and do actually contain a vfat filesystem with the appropriate files one would expect in the ESP.
The above behavior seems like a bug in the installer.
Workaround:
Since I had the system booted successfully from the SATA drives, I did the following:
- Verified the partition layout was identical on the NVMe and SATA drives.
- Wiped p2 on both NVMe drives to remove any trace of mdadm. Required a reboot afterwards.
- Reran pve-efiboot-tool to format and init those partitions.
- Added p3 (root) partitions from the NVMe as mirrors to zfs rpool, and detached the SATA drives from the pool.
- Rebooted to verify booting was working from the NVMe drives.
- Wiped the SATA drive partitions (sgdisk --zap-all)
- Verified that it still booted.
- Logged in via the GUI, verified that it looks like it's working as expected.
- Are there any issues with what I did to get the system online? Did I miss any critical steps?
- Does PVE know how to manage the two ESP partitions on two different boot drives?
- Am I likely to have any future issues with kernel or initram updates, or anything that might cause boot issues?
Thanks.
