BOTH grub + uefi boot/install possible ?

RolandK

RolandK

Famous Member
Proxmox Subscriber
Mar 5, 2019
1,027
226
108
52
hello,

i want to have proxmox system disks be interchangeable between different servers, e.g. to use spare hardware when production hardware has a defect.

the spare hardware can only do grub legacy boot, the production hardware is capable of uefi boot

how can i make sure that both boot grub & uefi boot is being setup on install or how can i force "grub only" at installation time ?
 
While this is far easier to accomplish with linux than with other operating systems (the simply changing the hardware in case of defect) - keep in mind that it seldomly really is a replacment by swapping the disks (since a very long while you at least need to check/adapt the network interface names).

apart from that legacy boot only hardware is luckily being phased out (and it took decades) - so on a hunch I'd also ask if your spare hardware would really be able to handle the load of a newer box it tries to replace

I do assume you're talking about ZFS on root systems? (since you're using uefi and grub as the 2 different modes, instead of e.g. uefi and legacy, or grub and systemd-boot).
If not (and without trying) - it should be enough to install grub in both modes (i386-pc and x86_64-efi) on the same disk (if it's correctly partitioned) and be somewhat happy

for the proxmox-boot-tool systems there was a patch-series a while ago -
https://lists.proxmox.com/pipermail/pve-devel/2022-February/051755.html
but IIRC we did not pursue it further since it might clash with plans to add secure-boot to PVE...

RolandK said:
or how can i force "grub only" at installation time ?
Click to expand...
boot the system in legacy mode and run the installer - that should work
 
  • Like
Reactions: RolandK
Now that proxmox-boot-tool is used to keep multiple ESP partitions in sync, can't Proxmox just use GRUB (using the ESP) for ZFS root pool like all other configurations? The two possible boot loaders is confusing to many people, makes troubleshooting and helping people more complex and causes extra works for the Proxmox developers. But I don't want to hijack this thread for that. Sorry, couldn't resist.
 
  • Like
Reactions: RolandK
leesteken said:
Now that proxmox-boot-tool is used to keep multiple ESP partitions in sync, can't Proxmox just use GRUB (using the ESP) for ZFS root pool like all other configurations? The two possible boot loaders is confusing to many people, makes troubleshooting and helping people more complex and causes extra works for the Proxmox developers. But I don't want to hijack this thread for that. Sorry, couldn't resist.
Click to expand...
We also considered that - and as you know, since you're helping many users with these issues here - once the esps are involved via p-b-t part of the complexity is already present (e.g. just because /boot/grub/grub.cfg has a certain content it doesn't mean that this config is the one that is used for booting).

one point that speaks for systemd-boot is that it's quite a bit simpler and less burdened by years of quirks being added to it for certain edge-cases...

In any case I think we'll change something regarding the choice of boot-loaders to make the situation simpler (the two options that come to mind currently are - grub for legacy systems only, systemd-boot for uefi systems (for all deployments) - or grub for all systems (there also was a patch-series for that)) - but it's still a bit dependent on some other decisions (e.g. how we address secure-boot)

I hope this explains it!
 
  • Like
Reactions: RolandK and leesteken
thank you for the infos @Stoiko Ivanov

>While this is far easier to accomplish with linux than with other operating systems (the simply changing the hardware
>in case of defect) - keep in mind that it seldomly really is a replacment by swapping the disks (since a very long while
>you at least need to check/adapt the network interface names).

yes, but it's really simple to handle.

>apart from that legacy boot only hardware is luckily being phased out (and it took decades) - so on a hunch
> I'd also ask if your spare hardware would really be able to handle the load of a newer box it tries to replace

yes it is, as our production hardware also is not brand new and furthermore we are running production and testing virtual machines on the same hardware, so we can just leave the testing vms powered down on the older, less powerful hardware.

>I do assume you're talking about ZFS on root systems? (since you're using uefi
>and grub as the 2 different modes, instead of e.g. uefi and legacy, or grub and systemd-boot).

yes

>for the proxmox-boot-tool systems there was a patch-series a while ago -
>https://lists.proxmox.com/pipermail/pve-devel/2022-February/051755.html
>but IIRC we did not pursue it further since it might clash with plans to add secure-boot to PVE...

ah ok. yes, it was confusing that grub is being installed on partition1 (which is not transparent) for legacy boot and on partition2 for uefi/systemd boot (where you can see appropriate files in the efi dir)


>leesteken:
>The two possible boot loaders is confusing to many people, makes troubleshooting and helping people
>more complex and causes extra works for the Proxmox developers.

yes, this is making things difficult.

i think that adding more transparency to WHAT proxmox-boot-tool DOES or HOW it works could be an improvement. also adding option to install grub or efi bootloader would be helpful.

for example, i was TOTALLY confused for a long time what bios-boot partition 1 is being used for at all, as i neither tell proxmox-boot-tool to touch it in any way , nor does proxmox-boot-tool report, that it (our some sub processing) would touch partition #1 at all. i was wondering, how boot could work when partition 1 isn't getting touched - but it is.

i have never seen a partitioning/formatting tool or any tool which modifies block-device contents would touching partitions besides the one(s) specified.

proxmox-boot-tool does.

i think at least some more verbosity would be an improvement and make things easier to understand


before: # xxd /dev/sdb1 |head 00000000: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000020: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000030: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000040: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000050: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000060: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000070: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000080: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000090: 0000 0000 0000 0000 0000 0000 0000 0000 ................ # proxmox-boot-tool format /dev/sdb2 UUID="" SIZE="536870912" FSTYPE="" PARTTYPE="c12a7328-f81f-11d2-ba4b-00a0c93ec93b" PKNAME="sdb" MOUNTPOINT="" Formatting '/dev/sdb2' as vfat.. mkfs.fat 4.2 (2021-01-31) Done. # proxmox-boot-tool init /dev/sdb2 Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace.. UUID="DE21-5960" SIZE="536870912" FSTYPE="vfat" PARTTYPE="c12a7328-f81f-11d2-ba4b-00a0c93ec93b" PKNAME="sdb" MOUNTPOINT="" Mounting '/dev/sdb2' on '/var/tmp/espmounts/DE21-5960'. Installing grub i386-pc target.. Installing for i386-pc platform. Installation finished. No error reported. Unmounting '/dev/sdb2'. Adding '/dev/sdb2' to list of synced ESPs.. Refreshing kernels and initrds.. Running hook script 'proxmox-auto-removal'.. Running hook script 'zz-proxmox-boot'.. Copying and configuring kernels on /dev/disk/by-uuid/A95B-5BAB Copying kernel 5.15.30-2-pve Generating grub configuration file ... Found linux image: /boot/vmlinuz-5.15.30-2-pve Found initrd image: /boot/initrd.img-5.15.30-2-pve done Copying and configuring kernels on /dev/disk/by-uuid/DE21-5960 Copying kernel 5.15.30-2-pve Generating grub configuration file ... Found linux image: /boot/vmlinuz-5.15.30-2-pve Found initrd image: /boot/initrd.img-5.15.30-2-pve done after: root@pve:~# xxd /dev/sdb1 |head 00000000: 5256 be1b 81e8 3901 5ebf f481 668b 2d83 RV....9.^...f.-. 00000010: 7d08 000f 84e2 0080 7cff 0074 4666 8b1d }.......|..tFf.. 00000020: 668b 4d04 6631 c0b0 7f39 4508 7f03 8b45 f.M.f1...9E....E 00000030: 0829 4508 6601 0566 8355 0400 c704 1000 .)E.f..f.U...... 00000040: 8944 0266 895c 0866 894c 0cc7 4406 0070 .D.f.\.f.L..D..p 00000050: 50c7 4404 0000 b442 cd13 0f82 af00 bb00 P.D....B........ 00000060: 70eb 6666 8b45 0466 09c0 0f85 9700 668b p.ff.E.f......f. 00000070: 0566 31d2 66f7 3488 540a 6631 d266 f774 .f1.f.4.T.f1.f.t 00000080: 0488 540b 8944 0c3b 4408 7d79 8b04 2a44 ..T..D.;D.}y..*D 00000090: 0a39 4508 7f03 8b45 0829 4508 6601 0566 .9E....E.)E.f..f
 
Last edited:
RolandK said:
Installing grub i386-pc target..
Installing for i386-pc platform.
Click to expand...
that's what's happening ;)

`proxmox-boot-tool init` simply installs the appropriate boot-loader to the disk to make it bootable...
in the legacy bios case that means grub with target i386-pc.
and this target is also the reason the first partition exists ...
https://wiki.archlinux.org/title/GRUB#GUID_Partition_Table_(GPT)_specific_instructions

so not quite sure what more of output would be helpful here?

I hope this explains it
 
  • Like
Reactions: RolandK
>so not quite sure what more of output would be helpful here?

it would be helpful to know that something is getting written to /dev/sdb or /dev/sdb1 when you where telling "please init /dev/sdb2". i find this totally difficult to follow/understand and it's sort of "magic" happening here with only that message being printed, especially when you find grub files on /dev/sdb2 afterwards, too - and when it's not easy to look at all places what's getting written. not everybody is able to use hexdump tool to inspect raw partition contents.
 
Last edited:
this is how grub works (or all legacy boot-loaders) - but I see that it could help mentioning in the docs that the `init` subcommand does install a boot-loader on the disk configured to boot from the esp (in our default partitioning schema the second partition).
 
  • Like
Reactions: RolandK
that would be very helpful, thanks!

this could perhaps also increase transparency

# diff -Naur /root/proxmox-boot-tool /usr/sbin/proxmox-boot-tool --- /root/proxmox-boot-tool 2022-09-15 11:47:59.008499099 +0200 +++ /usr/sbin/proxmox-boot-tool 2022-09-15 11:48:27.468305858 +0200 @@ -160,7 +160,7 @@ echo "default proxmox-*" >> "$esp_mp/$PMX_LOADER_CONF.tmp" mv "$esp_mp/$PMX_LOADER_CONF.tmp" "$esp_mp/$PMX_LOADER_CONF" else - echo "Installing grub i386-pc target" + echo "Installing grub i386-pc target on /dev/$PKNAME" grub-install.real \ --boot-directory "$esp_mp" \ --target i386-pc \
 
Last edited:
You must log in or register to reply here.
Top Bottom