minor: grub-probe error during update

[RolandK]

i installed a fresh proxmox 7.3 and during upgrade to latest pvetest, i noticed this grub error message.

i'm using zfs root. the system (futro s740) does not have legacy boot and can only boot via uefi

not sure if that's worth taking a closer look, but it looks ugly/weird.

i can offer debugging this, if desired. @Stoiko Ivanov , please let me know.

at the moment, i'm sensible about any s740 boot issues, because i had lost uefi boot entries yesterday. that also had been reported by a colleague of mine, who owns the same system.
at least for my system, it was a flaky sata connection and i think that was causing corruption on partitions (for example partition guid on /dev/sda2 suddenly had some weird value). anyhow, it's weird that uefi boot entry for proxmox got entirely lost from nvram (while i could still import rpool via proxmox boot cd).

Setting up grub-pc (2.06-3~deb11u5) ...
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.15.102-1-pve
Found initrd image: /boot/initrd.img-5.15.102-1-pve
/usr/sbin/grub-probe: error: unknown filesystem.      <---!
Found linux image: /boot/vmlinuz-5.15.74-1-pve
Found initrd image: /boot/initrd.img-5.15.74-1-pve
/usr/sbin/grub-probe: error: unknown filesystem.    <---!
Found memtest86+ image: /ROOT/pve-1@/boot/memtest86+.bin
Found memtest86+ multiboot image: /ROOT/pve-1@/boot/memtest86+_multiboot.bin
Warning: os-prober will not be executed to detect other bootable partitions.
Systems on them will not be added to the GRUB boot configuration.
Check GRUB_DISABLE_OS_PROBER documentation entry.
Adding boot menu entry for UEFI Firmware Settings ...
done
Setting up libproxmox-acme-plugins (1.4.4) ...
Setting up pve-kernel-5.15 (7.3-3) ...
Setting up swtpm (0.8.0~bpo11+3) ...
Setting up bind9-dnsutils (1:9.16.37-1~deb11u1) ...
Setting up libzpool5linux (2.1.9-pve1) ...
Setting up zfsutils-linux (2.1.9-pve1) ...
Setting up swtpm-tools (0.8.0~bpo11+3) ...
Setting up zfs-initramfs (2.1.9-pve1) ...
Setting up zfs-zed (2.1.9-pve1) ...
Setting up libproxmox-acme-perl (1.4.4) ...
Setting up libpve-common-perl (7.3-2) ...
Setting up pve-cluster (7.3-2) ...
Setting up libpve-cluster-perl (7.3-2) ...
Setting up libpve-http-server-perl (4.1-6) ...
Setting up libpve-storage-perl (7.3-2) ...
Setting up libpve-access-control (7.3-2) ...
Setting up libpve-cluster-api-perl (7.3-2) ...
Setting up qemu-server (7.3-4) ...
Setting up pve-manager (7.3-6) ...
Processing triggers for man-db (2.9.4-2) ...
Processing triggers for mailcap (3.69) ...
Processing triggers for pve-ha-manager (3.5.1) ...
Processing triggers for initramfs-tools (0.140) ...
update-initramfs: Generating /boot/initrd.img-5.15.102-1-pve
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
Copying and configuring kernels on /dev/disk/by-uuid/2BF2-A476
        Copying kernel and creating boot-entry for 5.15.102-1-pve
        Copying kernel and creating boot-entry for 5.15.74-1-pve
Processing triggers for libc-bin (2.31-13+deb11u5) ...
Processing triggers for rsyslog (8.2102.0-2+deb11u1) ...

Your System is up-to-date


Seems you installed a kernel update - Please consider rebooting
this node to activate the new kernel.

starting shell

 

[Stoiko Ivanov]

i installed a fresh proxmox 7.3 and during upgrade to latest pvetest, i noticed this grub error message.

i'm using zfs root. the system (futro s740) does not have legacy boot and can only boot via uefi

not sure if that's worth taking a closer look, but it looks ugly/weird.

i can offer debugging this, if desired. @Stoiko Ivanov , please let me know.

Thanks for the report! - This does not seem to be a general issue - at least I could not reproduce it here in a OVMF VM:
VM config:

bios: ovmf
boot: order=scsi0;ide2
cores: 1
cpu: host
efidisk0: local-zfs:vm-149-disk-0,efitype=4m,size=1M
ide2: ISO:iso/proxmox-ve_7.3-1.iso,media=cdrom,size=1082874K
machine: q35
memory: 4096
name: isotest-uefi
net0: virtio=.....
numa: 0
ostype: l26
parent: f73fresh
scsi0: local-zfs:vm-149-disk-1,discard=on,size=16G,ssd=1
scsihw: virtio-scsi-pci
serial0: socket
smbios1: uuid=.....
sockets: 1
vmgenid: .....

* Install PVE 7.3 from ISO with ZFS (RAID 0) as disk selection
* boot into the system - enable pvetest repository (disable enterprise repository)
* apt update; apt full-upgrade
the error-message does not show up

* How exactly did you setup the system?
* Did you perform any other steps apart from enabling pvetest and running apt update; apt full-upgrade? (also consider any ansible/puppet/$cfg-mgmt-tool changes)

e.g. which piece of software causes this output?

Your System is up-to-date


Seems you installed a kernel update - Please consider rebooting
this node to activate the new kernel.

starting shell

Thanks!

 

[RolandK]

i did nothing special during installation, the only difference from "normal" was :

- use 445gb instead of 447gb for size of the zfs raid0
- use zstd for compression

then i disabled enterprise repo via webgui and enabled pvetest

after that i upgraded via webgui/upgrade button

# blkid
/dev/sda2: UUID="2BF2-A476" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="1804c132-cba5-4121-9f00-9b9a6f4da0a8"
/dev/sda3: LABEL="rpool" UUID="5831168327316133531" UUID_SUB="8876623549909574865" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="7db2dc4c-81fa-45a7-bdb9-598c24f8d3cb"
/dev/sda1: PARTUUID="ba4a4c55-adc6-4ee8-ac5b-d85636be387f"

root@s740-1:~# proxmox-boot-tool status
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
System currently booted with uefi
2BF2-A476 is configured with: uefi (versions: 5.15.102-1-pve, 5.15.74-1-pve)

root@s740-1:~# proxmox-boot-tool refresh
Running hook script 'proxmox-auto-removal'..
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
Copying and configuring kernels on /dev/disk/by-uuid/2BF2-A476
    Copying kernel and creating boot-entry for 5.15.102-1-pve
    Copying kernel and creating boot-entry for 5.15.74-1-pve

root@s740-1:~# proxmox-boot-tool reinit
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
UUID="2BF2-A476" SIZE="536870912" FSTYPE="vfat" PARTTYPE="c12a7328-f81f-11d2-ba4b-00a0c93ec93b" PKNAME="sda" MOUNTPOINT=""
Mounting '/dev/disk/by-uuid/2BF2-A476' on '/var/tmp/espmounts/2BF2-A476'.
Installing systemd-boot..
Copied "/usr/lib/systemd/boot/efi/systemd-bootx64.efi" to "/var/tmp/espmounts/2BF2-A476/EFI/systemd/systemd-bootx64.efi".
Copied "/usr/lib/systemd/boot/efi/systemd-bootx64.efi" to "/var/tmp/espmounts/2BF2-A476/EFI/BOOT/BOOTX64.EFI".
Random seed file /var/tmp/espmounts/2BF2-A476/loader/random-seed successfully written (512 bytes).
Created EFI boot entry "Linux Boot Manager".
Configuring systemd-boot..
Unmounting '/dev/disk/by-uuid/2BF2-A476'.
Adding '/dev/disk/by-uuid/2BF2-A476' to list of synced ESPs..

# fdisk -l
Disk /dev/sda: 447.13 GiB, 480103981056 bytes, 937703088 sectors
Disk model: TOSHIBA THNSNJ48
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 36121E0C-7650-4E24-89D5-4C6CBA6B92F6

Device       Start       End   Sectors   Size Type
/dev/sda1       34      2047      2014  1007K BIOS boot
/dev/sda2     2048   1050623   1048576   512M EFI System
/dev/sda3  1050624 933232640 932182017 444.5G Solaris /usr & Apple ZFS

# zpool list -v

NAME                                             SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT

rpool                                            444G   115G   329G        -         -     0%    25%  1.00x    ONLINE  -

  ata-TOSHIBA_THNSNJ480PCS3_94GS1024TFLW-part3   444G   115G   329G        -         -     0%  25.8%      -    ONLINE

Command (? for help): p
Disk /dev/sda: 937703088 sectors, 447.1 GiB
Model: TOSHIBA THNSNJ48
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): 36121E0C-7650-4E24-89D5-4C6CBA6B92F6
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 937703054
Partitions will be aligned on 8-sector boundaries
Total free space is 4470414 sectors (2.1 GiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1              34            2047   1007.0 KiB  EF02
   2            2048         1050623   512.0 MiB   EF00
   3         1050624       933232640   444.5 GiB   BF01

Btw, what about setting proper Partition names when partitioning with gdisk ? ( https://gparted.org/display-doc.php?name=help-manual#gparted-name-partition )

 

[Stoiko Ivanov]

- use zstd for compression

pretty certain that this is the reason:
* as written in https://pve.proxmox.com/wiki/ZFS:_Switch_Legacy-Boot_to_Proxmox_Boot_Tool#Background - the ZFS implementation in grub does lack support for new features - and if such a feature, which requires write support is enabled this might cause the error-message (zstd compression is such a feature)

I managed to reproduce it (with both your settings in the installer)

As, for now, I think the issue is purely cosmetic (PVE/proxmox-boot-tool will not use grub to boot from rpool (it will always boot from the ESP)) - and as such don't know if it's worth fixing

You can remove grub2 from this system - this would make the message go away

I hope this helps!

 

[RolandK]

fantastic, thanks for explaining! glad to hear. sounds totally logical.

any reason (besides "nobody yet cared" ) why is grub2 being installed at all when it makes no sense on uefi-only system ?

it's a little bit confusing to have grub on a system which does not use grub for boot at all, especially when there is no easy/comfortable way to properly force installation of grub besides uefi (to make the harddisk portable, so i can unplug and boot on a legacy system)

i uninstalled all of these packages :

# dpkg --list|grep grub
ii  grub-common                          2.06-3~deb11u5                 amd64        GRand Unified Bootloader (common files)
ii  grub-efi-amd64-bin                   2.06-3~deb11u5                 amd64        GRand Unified Bootloader, version 2 (EFI-AMD64 modules)
ii  grub-pc                              2.06-3~deb11u5                 amd64        GRand Unified Bootloader, version 2 (PC/BIOS version)
ii  grub-pc-bin                          2.06-3~deb11u5                 amd64        GRand Unified Bootloader, version 2 (PC/BIOS modules)
ii  grub2-common                         2.06-3~deb11u5                 amd64        GRand Unified Bootloader (common files for version 2)

regarding the question on naming gpt partitions above - what about adding information like "unused / UEFI only system" to first partition, to show when it's not being populated/used/initialized ?

or what about keeping timestamp/log information in GPT partition names ( 36 chars in length) when proxmox-boot-tool touching /dev/sda1 or sda2 ? those could perhaps be helpful, when troubleshooting

 

[Stoiko Ivanov]

any reason (besides "nobody yet cared" ) why is grub2 being installed at all when it makes no sense on uefi-only system ?

a few (part of which might translate to "nobody yet cared" for some):
* There was no need to remove it - grub has been installed on most linux systems in the past 15 years - and rather seldomly caused issues simply by being present (quite a few issues with it as boot-loader - but that is quite normal in the field of working with all kinds of hardware...)

it's a little bit confusing to have grub on a system which does not use grub for boot at all, especially when there is no easy/comfortable way to properly force installation of grub besides uefi (to make the harddisk portable, so i can unplug and boot on a legacy system)

* There was a plan to keep grub there and make switching systems from UEFI to legacy boot possible (I sent a patch series once, and think I'll pick it up again soon for similar reasons) - and legacy systems use grub (at least for Proxmox)

regarding the question on naming gpt partitions above - what about adding information like "unused / UEFI only system" to first partition, to show when it's not being populated/used/initialized ?

hm - I do think in partition numbers - and never gave too much thought on partition names (when looking at a disk-layout lsblk usually tells me what I need to know - and the names in my systems just tend to run out of sync with the actual use) - in which context do you look at partition names regularly and gain some information from them?
(Not saying that adding partition names is not possible, or that we won't consider it - just asking where the benefits are)

or what about keeping timestamp/log information in GPT partition names ( 36 chars in length) when proxmox-boot-tool touching /dev/sda1 or sda2 ? those could perhaps be helpful, when troubleshooting

apart from the manual initialization - proxmox-boot-tool usually only touches those during apt upgrades - which in turn is logged into apt's terminal log

 

[RolandK]

a few (part of which might translate to "nobody yet cared" for some):
* There was no need to remove it - grub has been installed on most linux systems in the past 15 years - and rather seldomly caused issues simply by being present (quite a few issues with it as boot-loader - but that is quite normal in the field of working with all kinds of hardware...)

ok, that's an argument.
but mind that with updates, it's causing unnecessary downloads or being unecessarily executed, doing probes, causing irrelevant messages. i.e. : making things more complex and stealing time.

there is a LOT of time being stolen from us admins because things not being optimized.

i bet i have lost weeks of my lifetime already , waiting for servers finish to update or to boot.

that's especially worse if you troubleshoot problems and need to reboot a couple of times.

i simply like it when things getting optimized.

* There was a plan to keep grub there and make switching systems from UEFI to legacy boot possible (I sent a patch series once, and think I'll pick it up again soon for similar reasons) - and legacy systems use grub (at least for Proxmox)

i'd love such feature

hm - I do think in partition numbers - and never gave too much thought on partition names (when looking at a disk-layout lsblk usually tells me what I need to know - and the names in my systems just tend to run out of sync with the actual use) - in which context do you look at partition names regularly and gain some information from them?
(Not saying that adding partition names is not possible, or that we won't consider it - just asking where the benefits are)

parted -l , gdisk -l , lsblk -o partlabel and probably other tools will show those and i like having meta-information for objects which don't have other means of gaining information from, when they have been touched/written last (by whom and why).
just a quick idea...no need for picking that up.

apart from the manual initialization - proxmox-boot-tool usually only touches those during apt upgrades - which in turn is logged into apt's terminal log

yes, but that log most likely isn't available if you have an unbootable/broken system you try to recover (or ask yourself what happened). it could be useful to know if your boot env has been touched recently or not and if the fail of booting could be related to the last update(s).

 

[RolandK]

You can remove grub2 from this system - this would make the message go away

unfortunatly, grub is getting reinstalled with each kernel update. seems because it's a dependency for the package

root@s740:~# apt upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following NEW packages will be installed:
  grub-common grub-pc grub-pc-bin grub2-common os-prober pve-kernel-5.15.102-1-pve
The following packages will be upgraded:
  pve-firmware pve-kernel-5.15 pve-kernel-helper pve-qemu-kvm
4 upgraded, 6 newly installed, 0 to remove and 0 not upgraded.
Need to get 197 MB of archives.
After this operation, 430 MB of additional disk space will be used.
Do you want to continue? [Y/n] ^C

# apt-cache depends pve-kernel-5.15.85-1-pve
pve-kernel-5.15.85-1-pve
  Depends: busybox
    busybox-static
  Depends: initramfs-tools
 |Recommends: grub-pc
 |Recommends: grub-efi-amd64
 |Recommends: grub-efi-ia32
  Recommends: <grub-efi-arm64>
  Suggests: pve-firmware

 

[RolandK]

besides the zfs error it seems update-grub also errors on /dev/sda1 where no zfs resides. at least it erros two times. not sure if it probes zfs part 2 times...

root@pve-macmini3:~# grub-probe --target=fs_uuid --device /dev/sda1
grub-probe: error: unknown filesystem.
root@pve-macmini3:~# grub-probe --target=fs_uuid --device /dev/sda2
2705-CED6
root@pve-macmini3:~# grub-probe --target=fs_uuid --device /dev/sda3
grub-probe: error: unknown filesystem.

it seems that it can't neiter be fixed with GRUB_DISABLE_OS_PROBER=true or GRUB_OS_PROBER_SKIP_LIST=..... in /etc/default/grub

 

[RolandK]

with

apt-mark hold grub-common grub-efi-amd64-bin grub-pc grub-pc-bin grub2-common

the problem is resolved.

with that, grub won't automatically reinstall after removal