Converting To Secure Boot Fails, Boot disk not visible after switching Bios to SB

[HawHoo]

I have a 3 node cluster
All installed in the last couple weeks form a 8.1.2 ISO
For some reason one of the nodes got installed in Bios boot mode.
I followed instructions here: https://pve.proxmox.com/pve-docs/chapter-sysadmin.html#sysboot_secure_boot
The "proxmox-boot-tool init /dev/nvme0n1p2/sda2 grub" command finished without errors
but
efibootmgr -v fails the expected output

I am still able to boot in Bios mode but forcing to efi fails

The issue is that the drive does not show up in the bios boot menu when I switch to secure boot.

The Proxmox installer will see the Drive no problem
For testing I installed a second (ols small) NVME and with secure boot enabled and was able to install Proxmox on the secondary drive no issues

I am missing a step to make the nvme (931.5 GB) visible in secure boot mode.
I know I can "just" Reinstall proxmox but I'd prefer a solution that allows me to keep the customizations I have made

Systems are both installed with ZFS
Here is my drive layout booted form the "EFI test Proxmox" installed on the 2nd Drive (nvme1n1)

root@pve-03a-rtd:~# lsblk -o +FSTYPE
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS FSTYPE
nvme0n1 259:0 0 238.5G 0 disk
├─nvme0n1p1 259:1 0 1007K 0 part
├─nvme0n1p2 259:2 0 1G 0 part /boot/efi vfat
└─nvme0n1p3 259:3 0 237.5G 0 part LVM2_member
├─pve-swap 252:0 0 8G 0 lvm [SWAP] swap
├─pve-root 252:1 0 69.4G 0 lvm / ext4
├─pve-data_tmeta 252:2 0 1.4G 0 lvm
│ └─pve-data 252:4 0 141.2G 0 lvm
└─pve-data_tdata 252:3 0 141.2G 0 lvm
└─pve-data 252:4 0 141.2G 0 lvm
nvme1n1 259:4 0 931.5G 0 disk
├─nvme1n1p1 259:5 0 1007K 0 part
├─nvme1n1p2 259:6 0 1G 0 part vfat
└─nvme1n1p3 259:7 0 930.5G 0 part zfs_member
root@pve-03a-rtd:~# ^C
root@pve-03a-rtd:~#

With the time I have spent I could have backed up the whole cluster and rebuilt it.
Any advice would be appreciated
Matthias

 

[scyto]

what does the command efibootmgr -v output when you boot in BIOS mode without the new disk?

 

[HawHoo]

Hi @scyto
Thanks for helping a newbie out

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Thu Apr 4 22:10:50 HST 2024 on pts/0
root@pve-03:~# efibootmgr -v
EFI variables are not supported on this system.
root@pve-03:~#

 

[scyto]

NP when it comes to the inner workings of UEFI on linux i am also a noob.

The above would indicate that there is a fundamental Linux issue with your system and EFI, i am on an 8.0 system that was upgraded to 8.1 (no secure boot) and on mine produces the following, i am on grub not systemd.

root@pve1:~# efibootmgr -v
BootCurrent: 0000
Timeout: 1 seconds
BootOrder: 0000,0001,0003,0007,0005,0009
Boot0000* proxmox       HD(2,GPT,de159af4-f1a7-4b0d-a39d-000986476331,0x800,0x200000)/File(\EFI\proxmox\grubx64.efi)
Boot0001* UEFI OS       HD(2,GPT,de159af4-f1a7-4b0d-a39d-000986476331,0x800,0x200000)/File(\EFI\BOOT\BOOTX64.EFI)..BO
Boot0003* UEFI: PXE IPv4 Intel(R) Ethernet Controller I226-V    PciRoot(0x0)/Pci(0x1d,0x0)/Pci(0x0,0x0)/MAC(48210b589c45,1)/IPv4(0.0.0.00.0.0.0,0,0)..BO
Boot0005  UEFI: PXE IPv6 Intel(R) Ethernet Controller I226-V    PciRoot(0x0)/Pci(0x1d,0x0)/Pci(0x0,0x0)/MAC(48210b589c45,1)/IPv6([::]:<->[::]:,0,0)..BO
Boot0007* UEFI: PXE IPv4 Intel(R) Ethernet Controller I226-V    PciRoot(0x0)/Pci(0x1d,0x2)/Pci(0x0,0x0)/MAC(48210b57dfd7,1)/IPv4(0.0.0.00.0.0.0,0,0)..BO
Boot0009  UEFI: PXE IPv6 Intel(R) Ethernet Controller I226-V    PciRoot(0x0)/Pci(0x1d,0x2)/Pci(0x0,0x0)/MAC(48210b57dfd7,1)/IPv6([::]:<->[::]:,0,0)..BO

Now I assume you installed with ZFS right?

What do. the systemd checks in the secure boot conversion articles show?

i.e output of findmnt /and lsblk -o +FSTYPE

Is it possible when you did the original install your BIOS on that node was set to legacy (i.e UEFI disabled = and i don't mean secure boot disabled i mean full legacy BIOS mode) i ask because it seems you system is assuming no EFI on that original install but not on the new install... i don't understand why...

 

[HawHoo]

Hi @scyto

Correct I installed on ZFS
------------------------------------------------------------------------------------------------------------
root@pve-03:~# findmnt /
TARGET SOURCE FSTYPE OPTIONS
/ rpool/ROOT/pve-1 zfs rw,relatime,xattr,posixacl,casesensitive
root@pve-03:~#
------------------------------------------------------------------------------------------------------------

root@pve-03:~# lsblk -o +FSTYPE
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS FSTYPE
sda 8:0 0 500G 0 disk LVM2_member
├─GRP--NAS02--LUN--01-vm--304--disk--0 252:0 0 4M 0 lvm
└─GRP--NAS02--LUN--01-vm--304--disk--1 252:1 0 96G 0 lvm
zd0 230:0 0 1M 0 disk
zd16 230:16 0 8.5G 0 disk
zd32 230:32 0 127G 0 disk
├─zd32p1 230:33 0 1M 0 part
├─zd32p2 230:34 0 1G 0 part ext4
└─zd32p3 230:35 0 126G 0 part LVM2_member
zd48 230:48 0 256G 0 disk
├─zd48p1 230:49 0 1G 0 part vfat
├─zd48p2 230:50 0 2G 0 part ext4
└─zd48p3 230:51 0 252.9G 0 part LVM2_member
zd64 230:64 0 32G 0 disk
├─zd64p1 230:65 0 100M 0 part vfat
├─zd64p2 230:66 0 16M 0 part
├─zd64p3 230:67 0 31.3G 0 part ntfs
└─zd64p4 230:68 0 573M 0 part ntfs
zd80 230:80 0 1M 0 disk
zd96 230:96 0 4M 0 disk
zd112 230:112 0 8.5G 0 disk
nvme0n1 259:0 0 931.5G 0 disk
├─nvme0n1p1 259:1 0 1007K 0 part
├─nvme0n1p2 259:2 0 1G 0 part vfat
└─nvme0n1p3 259:3 0 930.5G 0 part zfs_member
root@pve-03:~#
------------------------------------------------------------------------------------------------------------
From reading i understand that 8.1 will install both bios and efi initially

Under disks for this node in the Gui it shows
Device Type Usage Size GPT
/def/nvme0n1 nvme Partitions 1 TB YES
/def/nvme0n1p2 Partition Bios Boot 1.03 MB YES
/def/nvme0n1p2 Partition EFI 1.07 GB YES
/def/nvme0n1p2 Partition ZFS 999.3 GB YES

So this looks like to me that PVE got installed with the "ability to boot via EFI
here is the listing of the EFI Partition:
------------------------------------------------------------------------------------------------------------
root@pve-03:~# mount | grep nvme
/dev/nvme0n1p2 on /mnt/test type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro)

root@pve-03:~# ls /mnt/test -lha
total 141M
drwxr-xr-x 3 root root 4.0K Dec 31 1969 .
drwxr-xr-x 4 root root 4 Apr 6 12:38 ..
-rwxr-xr-x 1 root root 6.2K Apr 2 23:44 db
-rwxr-xr-x 1 root root 3.7K Apr 2 23:44 dbx
drwxr-xr-x 5 root root 4.0K Apr 4 21:25 grub
-rwxr-xr-x 1 root root 58M Mar 29 21:57 initrd.img-6.5.11-8-pve
-rwxr-xr-x 1 root root 58M Mar 29 22:01 initrd.img-6.5.13-3-pve
-rwxr-xr-x 1 root root 3.5K Apr 2 23:44 KEK
-rwxr-xr-x 1 root root 886 Apr 4 21:02 PK
-rwxr-xr-x 1 root root 13M Jan 30 02:27 vmlinuz-6.5.11-8-pve
-rwxr-xr-x 1 root root 13M Mar 20 00:45 vmlinuz-6.5.13-3-pve
root@pve-03:~# ^C
root@pve-03:~#
------------------------------------------------------------------------------------------------------------

So it looks to me that
a) grub and EFI (capability) are installed.
b) uefi is not seeing the "boot option" for efi on the disk and therefore is not reporting the Disk as bootable in UEFI only mode ?

 

[scyto]

You cant AFAIK just manually make the mount points - that doesn't mean anything.
This was all done automatically on my machine.



I had issues converting a grub/efi system to grub/efi/secure boot system -grub didn't do what it was supposed to as per the docs

for me i had to run this https://forum.proxmox.com/threads/switching-existing-grub-efi-to-secure-boot.144573/post-650985 i know your scenario is different, but i don't see any sign that grub did what it was supposed to.

also consider running the boot tool init command again?

 

[fabian]

you first need to install EFI grub on all ESPs manually. then switch to EFI boot and boot from those ESPs. then I would install the secure boot packages (if you don't already have them installed) and just re-init all the ESPs using proxmox-boot-tool . then you should be able to enable secure boot as well.