Boot fails and goes into busybox After PVE upgrade 7 to 8

dnut

Active Member
May 6, 2016
23
6
43
41
Hi,
I'm having issues at boot with a Proxmox Node after I upgraded from PVE 7 to 8.

The boot fails and goes into busybox.

But, if I stop the boot at Grub for a while, then try to boot, it works properly.
It seems I'm having the issue described here: https://pve.proxmox.com/wiki/ZFS:_Tips_and_Tricks#Boot_fails_and_goes_into_busybox

As described, I tried to apply the following configuration
edit /etc/default/grub and add "rootdelay=10" at GRUB_CMDLINE_LINUX_DEFAULT (i.e. GRUB_CMDLINE_LINUX_DEFAULT="rootdelay=10 quiet") and then issue a # update-grub

But when issuing the update-grub the system report this:
Bash:
root@pve:~# update-grub
Generating grub configuration file ...
W: This system is booted via proxmox-boot-tool:
W: Executing 'update-grub' directly does not update the correct configs!
W: Running: 'proxmox-boot-tool refresh'

Copying and configuring kernels on /dev/disk/by-uuid/3CF0-B5E6
        Copying kernel 5.15.131-2-pve
        Copying kernel 6.5.11-7-pve
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-6.5.11-7-pve
Found initrd image: /boot/initrd.img-6.5.11-7-pve
Found linux image: /boot/vmlinuz-5.15.131-2-pve
Found initrd image: /boot/initrd.img-5.15.131-2-pve
Warning: os-prober will not be executed to detect other bootable partitions.
Systems on them will not be added to the GRUB boot configuration.
Check GRUB_DISABLE_OS_PROBER documentation entry.
done
Copying and configuring kernels on /dev/disk/by-uuid/3D9A-BBD8
        Copying kernel 5.15.131-2-pve
        Copying kernel 6.5.11-7-pve
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-6.5.11-7-pve
Found initrd image: /boot/initrd.img-6.5.11-7-pve
Found linux image: /boot/vmlinuz-5.15.131-2-pve
Found initrd image: /boot/initrd.img-5.15.131-2-pve
Warning: os-prober will not be executed to detect other bootable partitions.
Systems on them will not be added to the GRUB boot configuration.
Check GRUB_DISABLE_OS_PROBER documentation entry.
done
Found linux image: /boot/vmlinuz-6.5.11-7-pve
Found initrd image: /boot/initrd.img-6.5.11-7-pve
/usr/sbin/grub-probe: error: unknown filesystem.
Found linux image: /boot/vmlinuz-5.15.131-2-pve
Found initrd image: /boot/initrd.img-5.15.131-2-pve
Found linux image: /boot/vmlinuz-5.15.39-3-pve
Found initrd image: /boot/initrd.img-5.15.39-3-pve
Found linux image: /boot/vmlinuz-5.4.195-1-pve
Found initrd image: /boot/initrd.img-5.4.195-1-pve
Found linux image: /boot/vmlinuz-5.4.73-1-pve
Found initrd image: /boot/initrd.img-5.4.73-1-pve
/usr/sbin/grub-probe: error: unknown filesystem.
Found memtest86+x64 image: /ROOT/pve-1@/boot/memtest86+x64.bin
Warning: os-prober will not be executed to detect other bootable partitions.
Systems on them will not be added to the GRUB boot configuration.
Check GRUB_DISABLE_OS_PROBER documentation entry.
done

I can't understand if I have to expect that the /etc/default/grub modification I did it's then used by 'proxmox-boot-tool refresh' in some manner.
Or, otherwise, I should proceed with the second Choice indicated by the guide:
edit /etc/default/zfs, set ZFS_INITRD_PRE_MOUNTROOT_SLEEP='4', and then issue a "update-initramfs -k 6.5.11-7-pve -u"

Any suggestions?

Thank you

Bash:
lsblk

NAME      MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda         8:0    0   3.5T  0 disk
├─sda1      8:1    0   3.5T  0 part
└─sda9      8:9    0     8M  0 part
sdb         8:16   0   3.5T  0 disk
├─sdb1      8:17   0   3.5T  0 part
└─sdb9      8:25   0     8M  0 part
sdc         8:32   0 223.6G  0 disk
├─sdc1      8:33   0  1007K  0 part
├─sdc2      8:34   0   512M  0 part
└─sdc3      8:35   0 223.1G  0 part
sdd         8:48   0 223.6G  0 disk
├─sdd1      8:49   0  1007K  0 part
├─sdd2      8:50   0   512M  0 part
└─sdd3      8:51   0 223.1G  0 part
sde         8:64   0   1.8T  0 disk /mnt/bck_usb
sdf         8:80   0 465.7G  0 disk
└─sdf1      8:81   0 465.7G  0 part
sdg         8:96   0   1.8T  0 disk
├─sdg1      8:97   0    16M  0 part
└─sdg2      8:98   0   1.8T  0 part
zd0       230:0    0   256G  0 disk
├─zd0p1   230:1    0   100M  0 part
├─zd0p2   230:2    0    16M  0 part
├─zd0p3   230:3    0 255.3G  0 part
└─zd0p4   230:4    0   568M  0 part
zd16      230:16   0   512G  0 disk
├─zd16p1  230:17   0   549M  0 part
└─zd16p2  230:18   0 511.5G  0 part
zd32      230:32   0     4M  0 disk
zd48      230:48   0   256G  0 disk
├─zd48p1  230:49   0   549M  0 part
└─zd48p2  230:50   0 255.5G  0 part
zd64      230:64   0   512G  0 disk
├─zd64p1  230:65   0   549M  0 part
└─zd64p2  230:66   0 511.5G  0 part
zd80      230:80   0     1M  0 disk
zd96      230:96   0     4M  0 disk
zd112     230:112  0     1M  0 disk
zd128     230:128  0   256G  0 disk
├─zd128p1 230:129  0   100M  0 part
├─zd128p2 230:130  0    16M  0 part
├─zd128p3 230:131  0 255.3G  0 part
└─zd128p4 230:132  0   568M  0 part


Package versions
Bash:
proxmox-ve: 8.1.0 (running kernel: 6.5.11-7-pve)
pve-manager: 8.1.3 (running version: 8.1.3/b46aac3b42da5d15)
proxmox-kernel-helper: 8.1.0
pve-kernel-5.15: 7.4-9
proxmox-kernel-6.5: 6.5.11-7
proxmox-kernel-6.5.11-7-pve-signed: 6.5.11-7
pve-kernel-5.4: 6.4-19
pve-kernel-5.15.131-2-pve: 5.15.131-3
pve-kernel-5.15.39-3-pve: 5.15.39-3
pve-kernel-5.4.195-1-pve: 5.4.195-1
pve-kernel-5.4.73-1-pve: 5.4.73-1
ceph-fuse: 17.2.7-pve1
corosync: 3.1.7-pve3
criu: 3.17.1-2
glusterfs-client: 10.3-5
ifupdown: not correctly installed
ifupdown2: 3.2.0-1+pmx7
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-4
libknet1: 1.28-pve1
libproxmox-acme-perl: 1.5.0
libproxmox-backup-qemu0: 1.4.1
libproxmox-rs-perl: 0.3.3
libpve-access-control: 8.0.7
libpve-apiclient-perl: 3.3.1
libpve-common-perl: 8.1.0
libpve-guest-common-perl: 5.0.6
libpve-http-server-perl: 5.0.5
libpve-network-perl: 0.9.5
libpve-rs-perl: 0.8.7
libpve-storage-perl: 8.0.5
libqb0: 1.0.5-1
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 5.0.2-4
lxcfs: 5.0.3-pve4
novnc-pve: 1.4.0-3
proxmox-backup-client: 3.1.2-1
proxmox-backup-file-restore: 3.1.2-1
proxmox-kernel-helper: 8.1.0
proxmox-mail-forward: 0.2.2
proxmox-mini-journalreader: 1.4.0
proxmox-offline-mirror-helper: 0.6.3
proxmox-widget-toolkit: 4.1.3
pve-cluster: 8.0.5
pve-container: 5.0.8
pve-docs: 8.1.3
pve-edk2-firmware: 4.2023.08-2
pve-firewall: 5.0.3
pve-firmware: 3.9-1
pve-ha-manager: 4.0.3
pve-i18n: 3.1.5
pve-qemu-kvm: 8.1.2-6
pve-xtermjs: 5.3.0-3
qemu-server: 8.0.10
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.2.2-pve1
 
Hey,
Update your defaut grub timeout of 6 at 8 sec, and it gonna do the job ;)
(Not working ? Increase more time :) it s hardware initializing time prblm )
 
Hey,
Update your defaut grub timeout of 6 at 8 sec, and it gonna do the job ;)
(Not working ? Increase more time :) it s hardware initializing time prblm )
Ok, just did it both rootdelay=10 and GRUB_TIMEOUT=10

Here's the relevant part of /etc/default/grub
Code:
GRUB_DEFAULT=0
GRUB_TIMEOUT=10
GRUB_DISTRIBUTOR="Proxmox Virtual Environment"
GRUB_CMDLINE_LINUX_DEFAULT="rootdelay=10 quiet"
GRUB_CMDLINE_LINUX="root=ZFS=rpool/ROOT/pve-1 boot=zfs"

I'ts a production server so I can't test the reboot process right now

Thank you
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!