Need to regenerate grub at each reboot

[jcarnus]

Hi,
2 days ago, I perform an apt-get dist-upgrade to update to latest PVE kernel (4.15.17.13) and since I am not able to reboot the machine anymore
I have the follozing error apparing in grub
----------
error: attempt to read or write outside of disk "hd1"
loading initial amdisk
error: you need to load the kerner first
-----------
I've managed to restore my machine by regenerating grub into disk (by using something similar to this)
https://extremeshok.com/7039/proxmox-grub-zfs-recovery-rescue-system-chroot-and-recover-grub/
but I still have a very uncommon thing, each time I reboot the machine, I need to regenerate the grub, otherwise I fall back to the previous message at the beginning of this post

IIf someone already has this issue, thanks

 

[wolfgang]

Hi,

are you using SSD as a rpool disk?
If so maybe your disk can't write anymore, because the wareout is reached.

 

[jcarnus]

Hi
No this is hdd drive

 

[wolfgang]

The error message does not match to this problem but it is a try worth.
Try to increase the boot delay
https://pve.proxmox.com/wiki/ZFS:_Tips_and_Tricks#Grub_boot_ZFS_problem

 

[jcarnus]

Hi,
Thanks but it doesnt change something, still the same error after a reboot. Is it the first time you listen of this?

 

[wolfgang]

Yes, I never see this before.

 

[jcarnus]

Ok, this is exactly what I used to generate the boot loader

zpool import -R /mnt rpool
mount -t proc /proc /mnt/proc
mount --rbind /dev /mnt/dev
mount --rbind /sys /mnt/sys

# Enter chroot
chroot /mnt /bin/bash
source /etc/profile

# Reinstall package
dpkg --configure -a
apt-get update && apt-get dist-upgrade -y
apt-get install --reinstall grub-pc
apt-get install --reinstall zfs-initramfs
apt-get install --reinstall pve-kernel-4.15.17-3 linux-image-amd64

# Reinstall grub
grub-install /dev/sda
grub-install /dev/sdb
update-grub2
for x in $(cat /proc/cmdline); do case $x in root=ZFS=*) BOOT=zfs; ;; esac; done
grub-probe /
update-initramfs -u -k all

# Exit chroot
exit
reboot

Do you see aby issue ? I dotn exactly understand what the for x is doing

 

[jcarnus]

For increasing delay, what line should be changed from

# If you change this file, run 'update-grub' afterwards to update
# /boot/grub/grub.cfg.
# For full documentation of the options in this file, see:
# info -f grub -n 'Simple configuration'

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="Proxmox Virtual Environment"
GRUB_CMDLINE_LINUX_DEFAULT="rootdelay=30 noquiet nosplash"
GRUB_CMDLINE_LINUX="root=ZFS=rpool/ROOT/pve-1 boot=zfs rootdelay=10 vga=normal n omodeset rootdelay=15"

# Disable os-prober, it might add menu entries for each guest
GRUB_DISABLE_OS_PROBER=true

# Uncomment to enable BadRAM filtering, modify to suit your needs
# This works with Linux (no patch required) and with any kernel that obtains
# the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...)
#GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef"

# Uncomment to disable graphical terminal (grub-pc only)
#GRUB_TERMINAL=console

# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command `vbeinfo'
#GRUB_GFXMODE=640x480

# Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux
#GRUB_DISABLE_LINUX_UUID=true

# Disable generation of recovery mode menu entries
GRUB_DISABLE_RECOVERY="true"

# Uncomment to get a beep at grub start
#GRUB_INIT_TUNE="480 440 1"

 

[wolfgang]

for x in $(cat /proc/cmdline); do case $x in root=ZFS=*) BOOT=zfs; ;; esac; done

This set the EVN $Boot to zfs. when ZFS=* is include in the LINUX Start CMD.

You have rootdelay three times in your config.
please use only in this lines.

GRUB_CMDLINE_LINUX_DEFAULT="quiet rootdelay=30"
GRUB_CMDLINE_LINUX="root=ZFS=rpool/ROOT/pve-1 boot=zfs nomodeset"

 

[jcarnus]

Is it not this boot variable causing the issue? ENV not showing it

 

[jcarnus]

Thanks, this solved the issue
Now the question, I never add one, just modified the first one, I have no idea why 3 threaa value has been updated during the apt-get dist-upgrade

 

[wolfgang]

I can only tell you Proxmox do not touch this.

 

[jcarnus]

Hi,
I just wanted to repopen this thread. Today I performed an apt-get update && apt-get dist-upgrade
During the process, everything went well and the kernel was updated from 4.15.17-13 to 4.15.17-14 . But after a reboot, I got the following issue with grub error compression algorithm inherit not supported.
I had to take the server under rescue to regenerate grub and all thing (exact script here), and they it was able to reboot
I used proxmox for multiple yera now and I never get any kernel update issue but the 2 last one, I got this king of issue. Do we have an idea on what is going on ?
Thanks

-----------
# Reboot the server under rescue-64pro in OVH
# From https://extremeshok.com/7039/proxmox-grub-zfs-recovery-rescue-system-chroot-and-recover-grub/

# Mount the filesystem under rescue
zpool import -R /mnt rpool
mount -t proc /proc /mnt/proc
mount --rbind /dev /mnt/dev
mount --rbind /sys /mnt/sys

# Enter chroot
chroot /mnt /bin/bash
source /etc/profile

# Reinstall package
dpkg --configure -a
apt-get update && apt-get dist-upgrade -y
apt-get install --reinstall grub-pc
apt-get install --reinstall zfs-initramfs
apt-get install --reinstall pve-kernel-4.15.17-3 linux-image-amd64

# Reinstall grub
grub-install /dev/sda
grub-install /dev/sdb
update-grub2
for x in $(cat /proc/cmdline); do case $x in root=ZFS=*) BOOT=zfs; ;; esac; done
grub-probe /
update-initramfs -u -k all

# Exit chroot
exit

reboot

 

[jcarnus]

Hi,
I have twice the same issue updating the kernel with 25 july release. I need to rewrite the grub each time I update the pve-kernel version. Am I the only to have this ?

 

[Joshua Laferriere]

Ok, this is exactly what I used to generate the boot loader

zpool import -R /mnt rpool
mount -t proc /proc /mnt/proc
mount --rbind /dev /mnt/dev
mount --rbind /sys /mnt/sys

# Enter chroot
chroot /mnt /bin/bash
source /etc/profile

# Reinstall package
dpkg --configure -a
apt-get update && apt-get dist-upgrade -y
apt-get install --reinstall grub-pc
apt-get install --reinstall zfs-initramfs
apt-get install --reinstall pve-kernel-4.15.17-3 linux-image-amd64

# Reinstall grub
grub-install /dev/sda
grub-install /dev/sdb
update-grub2
for x in $(cat /proc/cmdline); do case $x in root=ZFS=*) BOOT=zfs; ;; esac; done
grub-probe /
update-initramfs -u -k all

# Exit chroot
exit
reboot

Do you see aby issue ? I dotn exactly understand what the for x is doing

omg you saved my butt!

I'd like to mention that I got an error when running reinstall pve-kernel and this fixed it
aufs-dkms

as a side note. Don't try to install hadoop on the host. javajdk-1.8.0-jdk breaks lxc... and I had to do some crazy reinstallation of all my packages which lead me to this (failed zfs boot/grub) and this solution got me back up and running!

I had to install proxmox to another hd as non zfs, and then I was able to run the above commands.

Thank you again!