Newly mirrored ZFS drive not bootable on Secured Boot

[promoxer]

1. I installed PVE8 on a Secured Boot environment and ended up with GRUB. In my logs below, the bootable disk is 9701-213F
2. I'm trying to create a mirror by adding a 2nd drive to rpool
3. Adding to rpool was seamless, but the 2nd drive doesn't boot if I remove the 1st
4. The steps I tried are from https://www.thomas-krenn.com/de/wiki/Boot-Device_Replacement_-_Proxmox_ZFS_Mirror_Disk_austauschen
5. After all applicable steps, I get below, but my new disk isn't bootable.

root@pve:/scripts# proxmox-boot-tool status
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
System currently booted with uefi
9701-213F is configured with: grub (versions: 6.5.11-4-pve, 6.5.11-7-pve)
F4EC-2405 is configured with: uefi (versions: )
root@pve:/scripts#

6. After running proxmox-boot-tool a few more times, I ended up with below, but 7D22-7E68 still can't boot. Any ideas?

System currently booted with uefi
7D22-7E68 is configured with: uefi (versions: 6.5.11-4-pve, 6.5.11-7-pve)
9701-213F is configured with: grub (versions: 6.5.11-4-pve, 6.5.11-7-pve)

 

[Dunuin]

You mixed grub and uefi bootloaders. You have to choose the correct bootloader for your installation. As you are booted with uefi both disks probably should use uefi and not grub.

 

[promoxer]

1. I'm aware that it was stated System currently booted with uefi, however, my boot screen is a big blue one which was determined to be grub. I took this off the internet, can't reboot now.

2. And machine has secure boot enabled, trying to boot off the new mirrored disk gave a invalid signature error from secure boot.

3. I disabled secure boot, however, it still doesn't boot, just keeps jumping back to BIOS

4. That said, above are not my immediate problems. My problem is, my mirrored rpool can't boot, what should I do?

 

[promoxer]

And I don't remember having a choice of grub/uefi from PVE8, it was automatically selected for me. My previous machine PVE7 used the other one (not grub), and again, I don't remember being asked.

So conclusion: My current bootable drive is definitely using grub, I have no idea why proxmox-boot-tool status seems to indicate otherwise.

Big question: What steps do I need to make both drives in the mirrored rpool bootable OR how can I fix both to use uefi (assuming that this will make them bootable and bearing in mind I would like to have secure boot enabled)

 

[Dunuin]

And machine has secure boot enabled

Then its not using grub
And while PVE 8.1 is now freshly supporting secure boot, it is really annoying to use it. I would disable that unless you are forced to use it because of regulatory requirements.

And I don't remember having a choice of grub/uefi from PVE8, it was automatically selected for me. My previous machine PVE7 used the other one (not grub), and again, I don't remember being asked.

Thats not an option you will be asked for. If you boot that ISO using grub it will install grub. If you boot that ISO using systemd it will use uefi.
So you choose what to use in your UEFI (for exampel by enabling or disabling CSM).

 

[promoxer]

My BIOS CSM is definitely disabled (and secure boot enabled) + my boot screen definitely has the word GRUB on line 1.

Do you know the steps to make the newly added drive bootable?

 

[leesteken]

My BIOS CSM is definitely disabled (and secure boot enabled) + my boot screen definitely has the word GRUB on line 1.

What does Proxmox show in the node Summary page? It's on the right side of "Boot Mode".

Do you know the steps to make the newly added drive bootable?

Search the manual ( https://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_zfs ) for "Changing a failed bootable device". There are also Wiki pages and various threads about that on this forum.

 

[promoxer]

What does Proxmox show in the node Summary page? It's on the right side of "Boot Mode".

root@pve:/scripts# proxmox-boot-tool status
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
System currently booted with uefi
9701-213F is configured with: grub (versions: 6.5.11-4-pve, 6.5.11-7-pve)

I'm not sure what you are trying to drive at, and it might not be what you currently believe, but it is definitely grub.
To answer your question, the summary page says EFI (Secure Boot), but I'm not buying any of it if you going to use that and claim I'm not booting off grub.

Grub or not, I think the information provided in my OP indicates I'm aware and have already done the sgdisk, proxmox-boot-tool to make a disk bootable. I do not think re-reading the documentation is a good use of my time.

 

[promoxer]

root@pve:/scripts# proxmox-boot-tool status
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
System currently booted with uefi
9536-5FB5 is configured with: grub (versions: 6.5.11-4-pve, 6.5.11-7-pve)
9701-213F is configured with: grub (versions: 6.5.11-4-pve, 6.5.11-7-pve)
root@pve:/scripts#

The way to get the above is proxmox-boot-tool init /dev/sdb2 grub

 

[leesteken]

GRUB is not mutually exclusive with (U)EFI. You can boot in (U)EFI mode with GRUB and you can also have root on a ZFS pool. That last combination is non-standard but not necessarily a problem. As long a proxmox-boot-tool handles your ESP partitions you'll probably be fine and you can probably follow the same instructions to replace (or add) a bootable mirror to your rpool.

 

[promoxer]

Yes, quite a stark contrast from telling me its not using grub and the technically correct term of the 2 boot-loaders is GRUB and systemd-boot.

(U)EFI is not a boot-loader, several other misleading conversations were practically referring to systemd-boot as (U)EFI.

 

[_gabriel]

grub bootloader is used when Secure Boot is enabled, even with ZFS + EFI.

 

[leesteken]

grub bootloader is used when Secure Boot is enabled, even with ZFS + EFI.

Good to know, thank you. Is there some documentation could can point me to?
EDIT: Last I read was Proxmox wanting to move to systemd-boot but I'm fine with GRUB; I just wish they used only one bootloader as so many people get confused...

 

[_gabriel]

https://pve.proxmox.com/wiki/Host_Bootloader