Proxmox boot to initramfs after reboot

[Icefire]

So I took the plungue to test Promox for our new server instead of Hyper-V.
I had to reinstall a second time for the same issue so I guess there is something I didn't do right.

System is a Dell R730 with 16 disk plus a SAS9200-8e card with a 24 disk sas.

I first installed the Promox on the first 8 disk as a RAIDZ2
made the updates rebooted
Made a second pool of the 24 disk, RAIDZ2 8 disk then I made zpool add to the pool to have 3 vdev
Installed Windows Server 2022
I added the last 8 disk of the 16 disk as another RAIDZ2 vdev to the rpool
Made the Promox update today, kernel change so I rebooted.
System was just hanging until I type enter and initramfs came.
zpool status showed no pool.

/sbin/zpool import -N rpool
exit
it booted, so I added a rootdelay=10 to grub, hoping it would work but I might have a theory.. I'm almost newbie to linux so bear with me.

In the GUI, I see the 8st disks of the rpool have an BIOS Boot / EFI / ZFS partition the others 8 only have ZFS / ZFS reserved. could it be it?
The 24 disk also only have those partition.

 

[aaron]

What is your actual question? If the disks belonging to the rpool have a different partition layout than the ones who belong to other ZFS pools?

If so, then yes. These disks contain partitions to store the bootloader and kernel images that are booted and then look for the ZFS pools.

 

[Icefire]

Sorry, I might have mispoke while trying to show how I did it.

When I installed Proxmox, I did it on only 8 of the 16 disk with an RAIDZ2 (rpool)
Then, later, I added another vdev in the rpool to double the capacity.

Is it normal the 8 first drive show 3 partion (BIOS BOOT/EFI/ZFS) and the other 8 in the same pool only show (ZFS/ZFS Reserved) ?

Problem I had (have?) on reboot of proxmox, the system halt midway on initramfs and zpool status show no pool. I had to /sbin/zpool import -N rpool.

Trying to find why it stop booting. I didn't see any problem in dmesg so I added a rootdelay=10 in grub, we will see if it work.

 

[aaron]

Is it normal the 8 first drive show 3 partion (BIOS BOOT/EFI/ZFS) and the other 8 in the same pool only show (ZFS/ZFS Reserved) ?

Yes, if you add them just by doing a zpool add rpool <vdev type> <list of disks> then this will happen. This leaves you with all the disks of the first vdev to boot from as they contain the bootloader / EFI partition.

For completeness’s sake, if someone else might come across this in the future; You can add the disks in a way so they also contain the boot partition. Follow the "Changing a failed bootable device" in the Documentation, especially the sgdisk and proxmox-boot-tool parts. Do this for all the disks and then instead of the zpool replace, run zpool add rpool <vdev type> <disk1 part3> <disk2 part3> .... As always, ideally you don't use the /dev/sdX paths, but /dev/disk/by-id/<manufacturer>-<model>-<serial nr> paths.

Problem I had (have?) on reboot of proxmox, the system halt midway on initramfs and zpool status show no pool. I had to /sbin/zpool import -N rpool.

Trying to find why it stop booting. I didn't see any problem in dmesg so I added a rootdelay=10 in grub, we will see if it work.

Yeah, depending on the controllers and how the system boots, it can take a while for all disks to become available. Try a reboot and see if the 10-second delay is enough for the system to boot cleanly.