[SOLVED] Kernel Boot Parameters not working

ducs4rs

Well-Known Member
Apr 13, 2021
41
6
48
68
I am trying to set UEFI kernel boot parameters but its not working. I am running 8.4.19. efibootmgr -v output is:

Code:
root@pveprod:/boot# efibootmgr -v
BootCurrent: 0006
Timeout: 1 seconds
BootOrder: 0004,0006,0007,0001,0005
Boot0001* Hard Drive    BBS(HD,,0x0)..GO..NO..........A.0.S.0. .S.a.m.s.u.n.g. .S.S.D. .8.5.0. .E.V.O. .m.S.A.T.A. .2.5.0.G.B....................A........................1.N........>.;......^..Gd-.;.A..MQ..L.A.0.S.0. .S.a.m.s.u.n.g. .S.S.D. .8.5.0. .E.V.O. .m.S.A.T.A. .2.5.0.G.B........BO..NO..........A.0.S.1. .S.a.m.s.u.n.g. .S.S.D. .8.4.0. .E.V.O. .2.5.0.G.B. .m.S.A.T.A....................A........................1.N........>.;......^..Gd-.;.A..MQ..L.A.0.S.1. .S.a.m.s.u.n.g. .S.S.D. .8.4.0. .E.V.O. .2.5.0.G.B. .m.S.A.T.A........BO..NO........o.S.a.m.s.u.n.g. .S.S.D. .8.7.0. .E.V.O. .2.5.0.G.B....................A...........................>..Gd-.;.A..MQ..L.6.S.V.1.Z.N.R.0.1.A.2.6.5.6. .D. . . . ........BO..NO........o.S.T.2.8.0.0.0.N.M.0.0.0.C.-.3.W.M.1.0.3....................A...........................>..Gd-.;.A..MQ..L. . . . . . . . . . . . .X.Z.0.A.A.B.V.B........BO..NO........o.S.a.m.s.u.n.g. .S.S.D. .8.7.0. .E.V.O. .2.5.0.G.B....................A...........................>..Gd-.;.A..MQ..L.6.S.D.P.J.N.R.0.4.5.4.1.0.6. .P. . . . ........BO..NO..........S.a.m.s.u.n.g. .S.S.D. .8.7.0. .E.V.O. .2.5.0.G.B....................A........................1.N........>.;......H..Gd-.;.A..MQ..L.S.a.m.s.u.n.g. .S.S.D. .8.7.0. .E.V.O. .2.5.0.G.B........BO..NO..........S.a.m.s.u.n.g. .S.S.D. .8.7.0. .E.V.O. .2.5.0.G.B....................A........................1.N........>.;......H..Gd-.;.A..MQ..L.S.a.m.s.u.n.g. .S.S.D. .8.7.0. .E.V.O. .2.5.0.G.B........BO..NO........o.S.T.2.8.0.0.0.N.M.0.0.0.C.-.3.W.M.1.0.3....................A...........................>..Gd-.;.A..MQ..L. . . . . . . . . . . . .X.Z.0.A.4.D.H.B........BO..NO........a.M.u.s.h.k.i.n. .V.e.n.t.u.r.a. .U.l.t.r.a. .0....................A................................Gd-.;.A..MQ..L.0.0.0.0.0.0.0.0.0.0.1.F........BO..NO........[.S.S.K. .P.o.r.t.a.b.l.e. .S.S.D. .1.2.8. .X.1.1.1....................A..........................Gd-.;.A..MQ..L.A.B.C.D.E.F.A.7.8.3.6.2........BO
Boot0004* Network Card  BBS(Network,,0x0)..GO..NO.........I.B.A. .X.E. .(.X.5.5.0.). .S.l.o.t. .0.4.0.0. .v.2.4.1.0.........................rN.D+..,.\...........P..Gd-.;.A..MQ..L.I.B.A. .X.E. .(.X.5.5.0.). .S.l.o.t. .0.4.0.0. .v.2.4.1.0........BO..NO.........I.B.A. .X.E. .(.X.5.5.0.). .S.l.o.t. .0.4.0.1. .v.2.4.1.0.........................rN.D+..,.\...........P..Gd-.;.A..MQ..L.I.B.A. .X.E. .(.X.5.5.0.). .S.l.o.t. .0.4.0.1. .v.2.4.1.0........BO
Boot0005* CD/DVD Drive  BBS(CDROM,,0x0)..GO..NO........c.A.M.I. .V.i.r.t.u.a.l. .C.D.R.O.M.0. .1...0.0....................A.............................0..Gd-.;.A..MQ..L.A.A.A.A.B.B.B.B.C.C.C.C.1........BO
Boot0006* UEFI OS       HD(2,GPT,8ba9dcf5-3276-4678-87d2-6b02af34b687,0x800,0x100000)/File(\EFI\BOOT\BOOTX64.EFI)..BO
Boot0007* UEFI OS       HD(2,GPT,676c88e3-3cc0-4dc5-a7e9-a055489e10c5,0x800,0x100000)/File(\EFI\BOOT\BOOTX64.EFI)..BO

/etc/kernel/cmd is

Code:
root=ZFS=rpool/ROOT/pve-1 boot=zfs rootdelay=15 quiet intel_iommu=on iommu=pt pci=assign-busses pci=nomsi pci=noaer pcie_aspm=off console=ttyS1 console=tty1

but /proc/cmdline shows

Code:
initrd=\EFI\proxmox\6.8.12-15-pve\initrd.img-6.8.12-15-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs rootdelay=15 quiet intel_iommu=on iommu=pt console=ttyS1 console=tty1

I have run proxmox-boot-tool refresh but nothing.

Any suggestions?
 
If your system boots with systemd-boot then you should edit /etc/kernel/cmdline (not /etc/kernel/cmd). If your system boots with GRUB then you should edit /etc/default/grub. If your boot is under the control of proxmox-boot-tool then you need to also run proxmox-boot-tool refresh. The manual will help you determine which bootloader is used and whether it is under the control of proxmox-boot-tool: https://pve.proxmox.com/pve-docs/pve-admin-guide.html#sysboot
 
If your system boots with systemd-boot then you should edit /etc/kernel/cmdline (not /etc/kernel/cmd). If your system boots with GRUB then you should edit /etc/default/grub. If your boot is under the control of proxmox-boot-tool then you need to also run proxmox-boot-tool refresh. The manual will help you determine which bootloader is used and whether it is under the control of proxmox-boot-tool: https://pve.proxmox.com/pve-docs/pve-admin-guide.html#sysboot
I had a typo in my post. I did edit /etc/kernel/cmdline. I am using systemd-boot and have done proxmox-boot-tool refresh, but the parameters are not being set.
 
I have a mirrored rpool with two disks, when I run proxmox-boot-tool status I get this


Code:
proxmox-boot-tool status
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
System currently booted with uefi
799C-C349 is configured with: grub (versions: 6.5.13-6-pve, 6.8.12-15-pve, 6.8.12-20-pve, 6.8.12-30-pve)
8243-668D is configured with: uefi (versions: 6.5.13-6-pve, 6.8.12-13-pve, 6.8.12-15-pve)

blkid of the two UUID's


Code:
root@pveprod:/boot# blkid |grep -i 8243-668D
/dev/sdi2: UUID="8243-668D" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="8ba9dcf5-3276-4678-87d2-6b02af34b687"
root@pveprod:/boot# blkid |grep -i 799C-C349
/dev/sdj2: UUID="799C-C349" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="676c88e3-3cc0-4dc5-a7e9-a055489e10c5"

proxmox-boot-tool refresh output below


Code:
root@pveprod:/boot# proxmox-boot-tool refresh
Running hook script 'proxmox-auto-removal'..
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
Copying and configuring kernels on /dev/disk/by-uuid/799C-C349
        Copying kernel 6.5.13-6-pve
        Copying kernel 6.8.12-15-pve
        Copying kernel 6.8.12-20-pve
        Copying kernel 6.8.12-30-pve
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-6.8.12-30-pve
Found initrd image: /boot/initrd.img-6.8.12-30-pve
Found linux image: /boot/vmlinuz-6.8.12-20-pve
Found initrd image: /boot/initrd.img-6.8.12-20-pve
Found linux image: /boot/vmlinuz-6.8.12-15-pve
Found initrd image: /boot/initrd.img-6.8.12-15-pve
Found linux image: /boot/vmlinuz-6.5.13-6-pve
Found initrd image: /boot/initrd.img-6.5.13-6-pve
Warning: os-prober will not be executed to detect other bootable partitions.
Systems on them will not be added to the GRUB boot configuration.
Check GRUB_DISABLE_OS_PROBER documentation entry.
Adding boot menu entry for UEFI Firmware Settings ...
done

update-grub2 output


Code:
update-grub2
Generating grub configuration file ...
W: This system is booted via proxmox-boot-tool:
W: Executing 'update-grub' directly does not update the correct configs!
W: Running: 'proxmox-boot-tool refresh'

Copying and configuring kernels on /dev/disk/by-uuid/799C-C349
        Copying kernel 6.5.13-6-pve
        Copying kernel 6.8.12-15-pve
        Copying kernel 6.8.12-20-pve
        Copying kernel 6.8.12-30-pve
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-6.8.12-30-pve
Found initrd image: /boot/initrd.img-6.8.12-30-pve
Found linux image: /boot/vmlinuz-6.8.12-20-pve
Found initrd image: /boot/initrd.img-6.8.12-20-pve
Found linux image: /boot/vmlinuz-6.8.12-15-pve
Found initrd image: /boot/initrd.img-6.8.12-15-pve
Found linux image: /boot/vmlinuz-6.5.13-6-pve
Found initrd image: /boot/initrd.img-6.5.13-6-pve
Warning: os-prober will not be executed to detect other bootable partitions.
Systems on them will not be added to the GRUB boot configuration.
Check GRUB_DISABLE_OS_PROBER documentation entry.
Adding boot menu entry for UEFI Firmware Settings ...
done
/dev/disk/by-uuid/8243-668D contains no grub directory - skipping
Found linux image: /boot/vmlinuz-6.8.12-30-pve
Found initrd image: /boot/initrd.img-6.8.12-30-pve
/usr/sbin/grub-probe: error: unknown filesystem.
Found linux image: /boot/vmlinuz-6.8.12-20-pve
Found initrd image: /boot/initrd.img-6.8.12-20-pve
Found linux image: /boot/vmlinuz-6.8.12-15-pve
Found initrd image: /boot/initrd.img-6.8.12-15-pve
Found linux image: /boot/vmlinuz-6.5.13-6-pve
Found initrd image: /boot/initrd.img-6.5.13-6-pve
/usr/sbin/grub-probe: error: unknown filesystem.
Warning: os-prober will not be executed to detect other bootable partitions.
Systems on them will not be added to the GRUB boot configuration.
Check GRUB_DISABLE_OS_PROBER documentation entry.
Adding boot menu entry for UEFI Firmware Settings ...
done
 
Code:
proxmox-boot-tool status
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
System currently booted with uefi
799C-C349 is configured with: grub (versions: 6.5.13-6-pve, 6.8.12-15-pve, 6.8.12-20-pve, 6.8.12-30-pve)
8243-668D is configured with: uefi (versions: 6.5.13-6-pve, 6.8.12-13-pve, 6.8.12-15-pve)
Why is one ESP configured with GRUB? Maybe your system is booting from that one? Maybe re-format and re-init it to be sure it is also UEFI and systemd-boot?
but /proc/cmdline shows

Code:
initrd=\EFI\proxmox\6.8.12-15-pve\initrd.img-6.8.12-15-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs rootdelay=15 quiet intel_iommu=on iommu=pt console=ttyS1 console=tty1
The current kernel version does not help in distinguishing since that one is on both. It is weird that the GRUB one has newer kernel versions, which suggest that it has been refreshed more recently.
Code:
root@pveprod:/boot# proxmox-boot-tool refresh
Running hook script 'proxmox-auto-removal'..
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
Copying and configuring kernels on /dev/disk/by-uuid/799C-C349
        Copying kernel 6.5.13-6-pve
        Copying kernel 6.8.12-15-pve
        Copying kernel 6.8.12-20-pve
        Copying kernel 6.8.12-30-pve
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-6.8.12-30-pve
Found initrd image: /boot/initrd.img-6.8.12-30-pve
Found linux image: /boot/vmlinuz-6.8.12-20-pve
Found initrd image: /boot/initrd.img-6.8.12-20-pve
Found linux image: /boot/vmlinuz-6.8.12-15-pve
Found initrd image: /boot/initrd.img-6.8.12-15-pve
Found linux image: /boot/vmlinuz-6.5.13-6-pve
Found initrd image: /boot/initrd.img-6.5.13-6-pve
Warning: os-prober will not be executed to detect other bootable partitions.
Systems on them will not be added to the GRUB boot configuration.
Check GRUB_DISABLE_OS_PROBER documentation entry.
Adding boot menu entry for UEFI Firmware Settings ...
done
proxmox-boot-tool refreshes only one of the ESP partitions (which is weird in itself) and it's the one with GRUB. Either Secure Boot is enabled or your are booting in BIOS/legacy mode instead of UEFI (which is influenced by motherboard BIOS/UEFI settings).
I think you need to fix your boot process first so it is GRUB or systemd-boot on both partitions and then your kernel parameters will probably also be fixed.

PS: intel_iommu=on is no longer needed since kernel version 6.8.
 
A second look revealed the issue. Your system is currently booted in UEFI mode but only the GRUB ESP partition is updated.
Code:
proxmox-boot-tool status
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
System currently booted with uefi
799C-C349 is configured with: grub (versions: 6.5.13-6-pve, 6.8.12-15-pve, 6.8.12-20-pve, 6.8.12-30-pve)
8243-668D is configured with: uefi (versions: 6.5.13-6-pve, 6.8.12-13-pve, 6.8.12-15-pve)

Code:
root@pveprod:/boot# proxmox-boot-tool refresh
Running hook script 'proxmox-auto-removal'..
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
Copying and configuring kernels on /dev/disk/by-uuid/799C-C349
        Copying kernel 6.5.13-6-pve
        Copying kernel 6.8.12-15-pve
        Copying kernel 6.8.12-20-pve
        Copying kernel 6.8.12-30-pve
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-6.8.12-30-pve
Found initrd image: /boot/initrd.img-6.8.12-30-pve
Found linux image: /boot/vmlinuz-6.8.12-20-pve
Found initrd image: /boot/initrd.img-6.8.12-20-pve
Found linux image: /boot/vmlinuz-6.8.12-15-pve
Found initrd image: /boot/initrd.img-6.8.12-15-pve
Found linux image: /boot/vmlinuz-6.5.13-6-pve
Found initrd image: /boot/initrd.img-6.5.13-6-pve
Warning: os-prober will not be executed to detect other bootable partitions.
Systems on them will not be added to the GRUB boot configuration.
Check GRUB_DISABLE_OS_PROBER documentation entry.
Adding boot menu entry for UEFI Firmware Settings ...
done
Maybe try booting the system from the other/GRUB ESP partition (instead of the current UEFI one) or figure out why only that ESP partition is updated.
 
A second look revealed the issue. Your system is currently booted in UEFI mode but only the GRUB ESP partition is updated.

Maybe try booting the system from the other/GRUB ESP partition (instead of the current UEFI one) or figure out why only that ESP partition is updated.
I removed the disk that had grub, rebooted, ran proxmox-boot-tool refresh, It worked. I am going to format the pulled disk and recreate the rpool mirror and see what happens. Thanks. I will mark this as fixed.
 
  • Like
Reactions: leesteken
I would assume the drive is fine, including the ZFS mirror (run a scrub to be sure). Just reformat/reinit both ESP partitions using the proxmox-boot-tool format and proxmox-boot-tool init commands (and clean/remove the old partition UUIDs as a format gives it a new one) should probably be enough to get them aligned. I wonder what proxmox-boot-tool status would report afterwards.