Proxmox 7.2 EFI/ZFS Default Kernel Boot

Cataract2

New Member
Jan 16, 2021
12
0
1
40
I currently have Proxmox 7.2 installed with EFI boot and ZFS. So, my system utilizes systemd for boot which means changes in grub do nothing. I'm stuck on how I change the default kernel, which right now is 5.15 for Proxmox 7.2, back to 5.13 as I need that kernel for now so I can utilize Nvidia vGPU drivers to split the gpu among my VMs. I know I can manually select 5.13 during boot, but as this is a headless system that is not ideal.
 

Cataract2

New Member
Jan 16, 2021
12
0
1
40
Yeah, as the docs state you need to make the changes live by executing proxmox-boot-tool refresh
So I followed the doc and tried the proxmox-boot-tool kernel pin option to select the 5.13 kernel. I ran the refresh command with it and yet upon reboot I am seeing it select the 5.15 kernel still.
1653602058219.png
 
Apr 3, 2022
104
37
28
So, it went something like this?

Code:
root@pve:~# proxmox-boot-tool kernel pin 5.13.19-6-pve
Set kernel '5.13.19-6-pve' in /etc/kernel/proxmox-boot-pin.
Refresh the actual boot ESPs now? [yN] y
Running hook script 'proxmox-auto-removal'..
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
Copying and configuring kernels on /dev/disk/by-uuid/9846-C1B1
        Copying kernel and creating boot-entry for 5.13.19-6-pve
        Copying kernel and creating boot-entry for 5.15.30-2-pve
        Copying kernel and creating boot-entry for 5.15.35-1-pve
root@pve:~#
 

Cataract2

New Member
Jan 16, 2021
12
0
1
40
So, it went something like this?

Code:
root@pve:~# proxmox-boot-tool kernel pin 5.13.19-6-pve
Set kernel '5.13.19-6-pve' in /etc/kernel/proxmox-boot-pin.
Refresh the actual boot ESPs now? [yN] y
Running hook script 'proxmox-auto-removal'..
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
Copying and configuring kernels on /dev/disk/by-uuid/9846-C1B1
        Copying kernel and creating boot-entry for 5.13.19-6-pve
        Copying kernel and creating boot-entry for 5.15.30-2-pve
        Copying kernel and creating boot-entry for 5.15.35-1-pve
root@pve:~#
Yes. It seems strange that it's not taking affect. Like it's not actually changing anything.
root@pve:~# proxmox-boot-tool kernel pin 5.13.19-6-pve
Set kernel '5.13.19-6-pve' in /etc/kernel/proxmox-boot-pin.
Refresh the actual boot ESPs now? [yN] y
Running hook script 'proxmox-auto-removal'..
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
Copying and configuring kernels on /dev/disk/by-uuid/E89E-57D3
Copying kernel and creating boot-entry for 5.13.19-6-pve
Copying kernel and creating boot-entry for 5.15.35-1-pve
Copying and configuring kernels on /dev/disk/by-uuid/E89F-21C3
Copying kernel and creating boot-entry for 5.13.19-6-pve
Copying kernel and creating boot-entry for 5.15.35-1-pve
root@pve:~# proxmox-boot-tool refresh
Running hook script 'proxmox-auto-removal'..
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
Copying and configuring kernels on /dev/disk/by-uuid/E89E-57D3
Copying kernel and creating boot-entry for 5.13.19-6-pve
Copying kernel and creating boot-entry for 5.15.35-1-pve
Copying and configuring kernels on /dev/disk/by-uuid/E89F-21C3
Copying kernel and creating boot-entry for 5.13.19-6-pve
Copying kernel and creating boot-entry for 5.15.35-1-pve
 

Stoiko Ivanov

Proxmox Staff Member
Staff member
May 2, 2018
6,979
1,082
164
Yes. It seems strange that it's not taking affect. Like it's not actually changing anything.
the change is in the loader.conf - mount the ESP (e.g. /dev/disk/by-uuid/E89F-21C3) and check $ESPMOUNTPOINT/loader/loader.conf
it should be the same on all your ESPs.

also make sure you are actually booting from one of the listed ESPs (and don't have some other ESP in the system)

I hope this helps!
 

Cataract2

New Member
Jan 16, 2021
12
0
1
40
the change is in the loader.conf - mount the ESP (e.g. /dev/disk/by-uuid/E89F-21C3) and check $ESPMOUNTPOINT/loader/loader.conf
it should be the same on all your ESPs.

also make sure you are actually booting from one of the listed ESPs (and don't have some other ESP in the system)

I hope this helps!
So, I mounted the ESP and checked the loader.conf. I show the kernel I pined in there.
1653697407679.png

So it is putting it in there. So I moved to the next part about which ESP might be in there. I do show both drives and I checks the ESPs on both in the loader.conf and both show the same as above.
1653697480017.png
Yet the boot still chooses the 5.15.
 
Last edited:

Stoiko Ivanov

Proxmox Staff Member
Staff member
May 2, 2018
6,979
1,082
164
Yet the boot still chooses the 5.15.
any chance you've set a version to be booted in an efi-var ? (usually with `bootctl` or inside the BIOS)?

else could you please post the complete output of `lsblk` (as text inside code tags)
 

Cataract2

New Member
Jan 16, 2021
12
0
1
40
any chance you've set a version to be booted in an efi-var ? (usually with `bootctl` or inside the BIOS)?

else could you please post the complete output of `lsblk` (as text inside code tags)
I don't believe I did. But here are the outputs of both bootctl and lsblk. Bios is set to just point to one of the two boot drives on booting.

Code:
root@pve:~# lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
loop0    7:0    0    50G  0 loop
loop1    7:1    0    60G  0 loop
sda      8:0    0   5.5T  0 disk
├─sda1   8:1    0   5.5T  0 part
└─sda9   8:9    0     8M  0 part
sdb      8:16   0   5.5T  0 disk
├─sdb1   8:17   0   5.5T  0 part
└─sdb9   8:25   0     8M  0 part
sdc      8:32   0   5.5T  0 disk
├─sdc1   8:33   0   5.5T  0 part
└─sdc9   8:41   0     8M  0 part
sdd      8:48   0   5.5T  0 disk
├─sdd1   8:49   0   5.5T  0 part
└─sdd9   8:57   0     8M  0 part
sde      8:64   0   5.5T  0 disk
├─sde1   8:65   0   5.5T  0 part
└─sde9   8:73   0     8M  0 part
sdf      8:80   0   1.8T  0 disk
├─sdf1   8:81   0   1.8T  0 part
└─sdf9   8:89   0     8M  0 part
sdg      8:96   0   1.8T  0 disk
├─sdg1   8:97   0   1.8T  0 part
└─sdg9   8:105  0     8M  0 part
sdh      8:112  0   1.8T  0 disk
├─sdh1   8:113  0   1.8T  0 part
└─sdh9   8:121  0     8M  0 part
sdi      8:128  0   1.8T  0 disk
├─sdi1   8:129  0   1.8T  0 part
└─sdi9   8:137  0     8M  0 part
sdj      8:144  0 465.8G  0 disk
├─sdj1   8:145  0  1007K  0 part
├─sdj2   8:146  0   512M  0 part
└─sdj3   8:147  0 465.3G  0 part
sdk      8:160  0 465.8G  0 disk
├─sdk1   8:161  0  1007K  0 part
├─sdk2   8:162  0   512M  0 part
└─sdk3   8:163  0 465.3G  0 part
sr0     11:0    1  1024M  0 rom

Code:
root@pve:~# bootctl
Couldn't find EFI system partition. It is recommended to mount it to /boot or /efi.
Alternatively, use --esp-path= to specify path to mount point.
System:
     Firmware: UEFI 2.70 (American Megatrends 5.17)
  Secure Boot: disabled
   Setup Mode: setup
 Boot into FW: supported

Current Boot Loader:
      Product: systemd-boot 241
     Features: ✓ Boot counting
               ✓ Menu timeout control
               ✓ One-shot menu timeout control
               ✓ Default entry control
               ✓ One-shot entry control
               ✗ Support for XBOOTLDR partition
               ✗ Support for passing random seed to OS
               ✓ Boot loader sets ESP partition information
WARNING: The boot loader reports different ESP UUID then detected!
          ESP: /dev/disk/by-partuuid/502b28d0-b228-4d2b-a967-0ae6ce4e9ed3
         File: └─/EFI/BOOT/BOOTX64.EFI

Random Seed:
 Passed to OS: no
 System Token: not set

Boot Loaders Listed in EFI Variables:
        Title: UEFI OS
           ID: 0x0007
       Status: active, boot-order
    Partition: /dev/disk/by-partuuid/502b28d0-b228-4d2b-a967-0ae6ce4e9ed3
         File: └─/EFI/BOOT/BOOTX64.EFI

        Title: UEFI OS
           ID: 0x0008
       Status: active, boot-order
    Partition: /dev/disk/by-partuuid/2c464620-9669-44c3-985f-f9a1505fab4d
         File: └─/EFI/BOOT/BOOTX64.EFI
 

Cataract2

New Member
Jan 16, 2021
12
0
1
40
could you try mounting and checking those 2 partitions?
Certainly, first one is E89F-21C3.
Code:
root@pve:/# mount /dev/disk/by-uuid/E89F-21C3 /boot/efi/
root@pve:/# bootctl
System:
     Firmware: UEFI 2.70 (American Megatrends 5.17)
  Secure Boot: disabled
   Setup Mode: setup
 Boot into FW: supported

Current Boot Loader:
      Product: systemd-boot 241
     Features: ✓ Boot counting
               ✓ Menu timeout control
               ✓ One-shot menu timeout control
               ✓ Default entry control
               ✓ One-shot entry control
               ✗ Support for XBOOTLDR partition
               ✗ Support for passing random seed to OS
               ✓ Boot loader sets ESP partition information
          ESP: /dev/disk/by-partuuid/502b28d0-b228-4d2b-a967-0ae6ce4e9ed3
         File: └─/EFI/BOOT/BOOTX64.EFI

Random Seed:
 Passed to OS: no
 System Token: not set
       Exists: no

Available Boot Loaders on ESP:
          ESP: /boot/efi (/dev/disk/by-partuuid/502b28d0-b228-4d2b-a967-0ae6ce4e9ed3)
         File: └─/EFI/systemd/systemd-bootx64.efi (systemd-boot 241)
         File: └─/EFI/BOOT/BOOTX64.EFI (systemd-boot 241)

Boot Loaders Listed in EFI Variables:
        Title: UEFI OS
           ID: 0x0007
       Status: active, boot-order
    Partition: /dev/disk/by-partuuid/502b28d0-b228-4d2b-a967-0ae6ce4e9ed3
         File: └─/EFI/BOOT/BOOTX64.EFI

        Title: UEFI OS
           ID: 0x0008
       Status: active, boot-order
    Partition: /dev/disk/by-partuuid/2c464620-9669-44c3-985f-f9a1505fab4d
         File: └─/EFI/BOOT/BOOTX64.EFI

Boot Loader Entries:
        $BOOT: /boot/efi (/dev/disk/by-partuuid/502b28d0-b228-4d2b-a967-0ae6ce4e9ed3)

Default Boot Loader Entry:
        title: Proxmox Virtual Environment (5.13.19-6-pve)
           id: proxmox-5.13.19-6-pve.conf
       source: /boot/efi/loader/entries/proxmox-5.13.19-6-pve.conf
      version: 5.13.19-6-pve
        linux: /EFI/proxmox/5.13.19-6-pve/vmlinuz-5.13.19-6-pve
       initrd: /EFI/proxmox/5.13.19-6-pve/initrd.img-5.13.19-6-pve
      options: root=ZFS=rpool/ROOT/pve-1 boot=zfs amd_iommu=on iommu=pt

Next one is E89E-57D3.
Code:
root@pve:/# mount /dev/disk/by-uuid/E89E-57D3 /boot/efi/
root@pve:/# bootctl
System:
     Firmware: UEFI 2.70 (American Megatrends 5.17)
  Secure Boot: disabled
   Setup Mode: setup
 Boot into FW: supported

Current Boot Loader:
      Product: systemd-boot 241
     Features: ✓ Boot counting
               ✓ Menu timeout control
               ✓ One-shot menu timeout control
               ✓ Default entry control
               ✓ One-shot entry control
               ✗ Support for XBOOTLDR partition
               ✗ Support for passing random seed to OS
               ✓ Boot loader sets ESP partition information
WARNING: The boot loader reports different ESP UUID then detected!
          ESP: /dev/disk/by-partuuid/502b28d0-b228-4d2b-a967-0ae6ce4e9ed3
         File: └─/EFI/BOOT/BOOTX64.EFI

Random Seed:
 Passed to OS: no
 System Token: not set
       Exists: no

Available Boot Loaders on ESP:
          ESP: /boot/efi (/dev/disk/by-partuuid/2c464620-9669-44c3-985f-f9a1505fab4d)
         File: └─/EFI/systemd/systemd-bootx64.efi (systemd-boot 241)
         File: └─/EFI/BOOT/BOOTX64.EFI (systemd-boot 241)

Boot Loaders Listed in EFI Variables:
        Title: UEFI OS
           ID: 0x0007
       Status: active, boot-order
    Partition: /dev/disk/by-partuuid/502b28d0-b228-4d2b-a967-0ae6ce4e9ed3
         File: └─/EFI/BOOT/BOOTX64.EFI

        Title: UEFI OS
           ID: 0x0008
       Status: active, boot-order
    Partition: /dev/disk/by-partuuid/2c464620-9669-44c3-985f-f9a1505fab4d
         File: └─/EFI/BOOT/BOOTX64.EFI

Boot Loader Entries:
        $BOOT: /boot/efi (/dev/disk/by-partuuid/2c464620-9669-44c3-985f-f9a1505fab4d)

Default Boot Loader Entry:
        title: Proxmox Virtual Environment (5.13.19-6-pve)
           id: proxmox-5.13.19-6-pve.conf
       source: /boot/efi/loader/entries/proxmox-5.13.19-6-pve.conf
      version: 5.13.19-6-pve
        linux: /EFI/proxmox/5.13.19-6-pve/vmlinuz-5.13.19-6-pve
       initrd: /EFI/proxmox/5.13.19-6-pve/initrd.img-5.13.19-6-pve
      options: root=ZFS=rpool/ROOT/pve-1 boot=zfs amd_iommu=on iommu=pt

From what I can see in these, everything seems to point to using kernel 5.13, but the efi "grub" during boot is defaulting itself to 5.15.
 

Stoiko Ivanov

Proxmox Staff Member
Staff member
May 2, 2018
6,979
1,082
164
* what's the output of `efibootmgr`?
* please try also running `bootctl set-default ''` , `bootctl set-oneshot ''` (to make sure the version is not pinned in a systemd efivar)
 

sterzy

Proxmox Staff Member
Staff member
Feb 1, 2022
216
51
28
Hi,

could you try to run proxmox-boot-tool init /dev/disk/by-partuuid/502b28d0-b228-4d2b-a967-0ae6ce4e9ed3 and see if that fixes the issue for you?

Thank you!
 
Last edited:

Javier Alonso

Member
Jul 12, 2020
2
1
8
33
Same here with UEFI and systemd-boot.

I tried proxmox-boot-tool kernel pin 5.13.19-6-pve and then proxmox-boot-tool refresh, but doesn't work.

Yesterday, i made a change in the /etc/kernel/cmdline in the same HV and then i run "proxmox-boot-tool refresh". After that the system boot up with 5.13.19-6-pve.
 

Javier Alonso

Member
Jul 12, 2020
2
1
8
33
Same here with UEFI and systemd-boot.

I tried proxmox-boot-tool kernel pin 5.13.19-6-pve and then proxmox-boot-tool refresh, but doesn't work.

Yesterday, i made a change in the /etc/kernel/cmdline in the same HV and then i run "proxmox-boot-tool refresh". After that the system boot up with 5.13.19-6-pve.
I have check the history and this is what works for me:

proxmox-boot-tool kernel pin 5.13.19-6-pve
proxmox-boot-tool reinit
proxmox-boot-tool refresh
 
  • Like
Reactions: Stoiko Ivanov

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!