[SOLVED] Proxmox kernel panic

B

borjaevo

New Member
Feb 1, 2022
19
2
3
37
I have a server running proxmox 5.4
The server is a proliant dual cpu and 64gb RAM. Today we had a power problem and the SAI coundnt provide power for so long.
After restarting it can no longer boot and throws a kernel panic
The error it shows is "Initramfs unpacking failed: no cpio magic"

I have no idea how can i solve the problem :(
 

Attachments

  • kernelPanic.png
    kernelPanic.png
    148.8 KB · Views: 180
First of. Your Proxmox version is End of Life. I highly encourage to upgrade to newer one for security reasons!

Next it appears like your initramfs has been corrupted. Do you have other kernels to choose from in your boot loader when you start? If not you will have to start your system from a live cd (for example Debian or Ubuntu).
 
If you need to fix your initramfs from a live cd try

Code: 
    sudo mkdir /media/RESCUE
    sudo mount /dev/pve/root /media/RESCUE/
    sudo mount -t proc proc /media/RESCUE/proc
    sudo mount -t sysfs sys /media/RESCUE/sys
    sudo mount -o bind /dev /media/RESCUE/dev
    sudo mount -o bind /run /media/RESCUE/run
    chroot /media/RESCUE
    update-initramfs -u
    exit
    reboot
 
Philipp Hufnagl said:
First of. Your Proxmox version is End of Life. I highly encourage to upgrade to newer one for security reasons!

Next it appears like your initramfs has been corrupted. Do you have other kernels to choose from in your boot loader when you start? If not you will have to start your system from a live cd (for example Debian or Ubuntu).
Click to expand...

Philipp Hufnagl said:
If you need to fix your initramfs from a live cd try

Code: 
    sudo mkdir /media/RESCUE
    sudo mount /dev/pve/root /media/RESCUE/
    sudo mount -t proc proc /media/RESCUE/proc
    sudo mount -t sysfs sys /media/RESCUE/sys
    sudo mount -o bind /dev /media/RESCUE/dev
    sudo mount -o bind /run /media/RESCUE/run
    chroot /media/RESCUE
    update-initramfs -u
    exit
    reboot
Click to expand...
Thank you!

Its a little bit more complicated because i have the SO partition in RAID1 ZFS. I tried to import the pool from Ubuntu but i couldn't. I use a USB flash drive to boot. The img initramfs is there...
 
In theory, after chroot you should be able to operate your operating system in the terminal like you have booted it normally

Are you sure you are looking at the right initramfs? Does it have *-pve in the name?

Even if it is there it might still be corrupted. You can move it to some other name and update after if you dont want to overwrite it.
 
Philipp Hufnagl said:
In theory, after chroot you should be able to operate your operating system in the terminal like you have booted it normally

Are you sure you are looking at the right initramfs? Does it have *-pve in the name?

Even if it is there it might still be corrupted. You can move it to some other name and update after if you dont want to overwrite it.
Click to expand...
This is what i have in the boot partition (picture attached). So i have the system in the ZFS RAID partition, i cannot mount the folders necesary to update initramfs. So i'm trying to import the zfs pools.
 

Attachments

  • bootPartition.png
    bootPartition.png
    68 KB · Views: 63
Im not sure what i am looking at. It appers that you mounted your boot partition to /mnt ? Did you chroot?

Can I see the output of lsblk and mount to get to know your system better?
 
Philipp Hufnagl said:
Im not sure what i am looking at. It appers that you mounted your boot partition to /mnt ? Did you chroot?

Can I see the output of lsblk and mount to get to know your system better?
Click to expand...
I understand that you are not sure about what are you looking at, because i also dont understand my own environment :D

My boot partition is on a USB flash drive, and it is supposed to load rpoool that is a ZFS raid 1 pool that contains the root filesystem. I achieved to mount this root filesystem partition on an ubuntu live. But i checked this partition and i have a /boot directory there that has an initrd image 5.13 insted of 5.4 so i don't understand...
I will try to boot directly into one of the disks that contains this ZFS volume so maybe it has a MBR capable of booting the ZFS root filesystem... im not sure.
Attached lsblk output picture.
 

Attachments

  • lsblk.png
    lsblk.png
    169.5 KB · Views: 27
I think what you see at /boot is not your proxmox but rather your ubuntu live system.

What are we trying to do is mounting your system to /media/RESCUE and then tell your shell your file system root (/) is changed to /media/RESCUE (thats what chroot does)

to do that we first have to mount important directories of your file system to /media/RESCUE
Code: 
sudo mkdir /media/RESCUE
sudo mount /dev/pve/root /media/RESCUE/
sudo mount /dev/my-usb /media/RESCUE/boot
sudo mount -t proc proc /media/RESCUE/proc
sudo mount -t sysfs sys /media/RESCUE/sys
sudo mount -o bind /dev /media/RESCUE/dev
sudo mount -o bind /run /media/RESCUE/run

Then we change the / in the terminal to /media/RESCUE
Code: 
chroot /media/RESCUE

lastly we can operate our proxmox as if we would have booted. We can for example regenerate the initramfs with
Code: 
update-initramfs -u

when we are done we can close our chroot environment and reboot with
Code: 
exit
reboot

If there is anything unclear please ask
 
Last edited:
Philipp Hufnagl said:
I think what you see at /boot is not your proxmox but rather your ubuntu live system.

What are we trying to do is mounting your system to /media/RESCUE and then tell your shell your file system root (/) is changed to /media/RESCUE (thats what chroot does)

to do that we first have to mount important directories of your file system to /media/RESCUE
Code: 
sudo mkdir /media/RESCUE
sudo mount /dev/pve/root /media/RESCUE/
sudo mount /dev/my-usb /media/RESCUE/boot
sudo mount -t proc proc /media/RESCUE/proc
sudo mount -t sysfs sys /media/RESCUE/sys
sudo mount -o bind /dev /media/RESCUE/dev
sudo mount -o bind /run /media/RESCUE/run

Then we change the / in the terminal to /media/RESCUE
Code: 
chroot /media/RESCUE

lastly we can operate our proxmox as if we would have booted. We can for example regenerate the initramfs with
Code: 
update-initramfs -u

when we are done we can close our chroot environment and reboot with
Code: 
exit
reboot

If there is anything unclear please ask
Click to expand...
Hi Philipp,

Yes, i understand. But so i have the root filesystem in ZFS and the boot partition in a flash drive outside the root filesystem i tried other things first. I replaced the img file of the boot partition with the ones in boot.old folder of root filesystem and i get the initramfs to load correctly. Problem now is is rpool is detecting is was last in used in other system. Not sure how is detecting that... the same message appeared in ubuntu when trying to mount it but you only have to use the -f option. Here, im not sure...
Picture attached
 

Attachments

  • rpooluse.png
    rpooluse.png
    181.8 KB · Views: 18
You are welcome. Please mark the thread as solved to make it easier for the next person to find
 
Also you should really consider upgrading! Your Proxmox version is End of Life
 
Philipp Hufnagl said:
If you need to fix your initramfs from a live cd try

Code: 
    sudo mkdir /media/RESCUE
    sudo mount /dev/pve/root /media/RESCUE/
    sudo mount -t proc proc /media/RESCUE/proc
    sudo mount -t sysfs sys /media/RESCUE/sys
    sudo mount -o bind /dev /media/RESCUE/dev
    sudo mount -o bind /run /media/RESCUE/run
    chroot /media/RESCUE
    update-initramfs -u
    exit
    reboot
Click to expand...
Hi! My Proxmox host fails to boot and stays stuck on "Loading initial RAMDisk". My host has been down for some time and I really do not want to wipe it. These suggestions look close to what I need to do to fix it, but my Proxmox install is located on a ZFS RAID 1 volume that I can import while running in the shell from the Proxmox install ISO. What do I need to change within these commands to make it work like you have suggested so that I can repair the kernel? Any help would be greatly appreciated!
 
You must log in or register to reply here.
Top Bottom