PVE 7.2 w/ Kernel 5.15 fails to boot past 'Loading initial ramdisk...'

M

Maurice

New Member
Mar 11, 2021
6
1
3
Hi all,

After apt update / upgrade my kernel went from 5.11.22-7-pve to 5.15.64-1-pve but now gets stuck at 'Loading initial ramdisk...'
Adding the nomodeset makes no difference, still no messages are shown.
Starting PVE with the 5.11.22-7-pve kernel works without issues.

I run PVE on:
- Supermicro SuperServer A2SDi-16C-HLN4F
- 16 Core Intel Atom C3955 (Denverton) processor
- 64GB DDR4 ECC memory
- 1TB Samsung 980 PRO (with Heatsink) M.2 NVMe SSD.

Any suggestions?
 
Hello,

The upgrade process went ok, i.e. did not appear errors during the upgrade process?

May I ask you what exactly command you used for the upgrade? It's an apt update && apt upgrade or apt update && dist-upgrade?

EDIT: Can you also provide us with the output of pveversion -v
 
Hi Moayad,
I have the same issue on a A2SDi-8C-HLN4F. Did apt update followed by apt dist-upgrade:

Here is the output you're looking for:
Code: 
proxmox-ve: 7.2-1 (running kernel: 5.11.22-7-pve)
pve-manager: 7.2-11 (running version: 7.2-11/b76d3178)
pve-kernel-5.15: 7.2-13
pve-kernel-helper: 7.2-13
pve-kernel-5.11: 7.0-10
pve-kernel-5.15.64-1-pve: 5.15.64-1
pve-kernel-5.11.22-7-pve: 5.11.22-12
pve-kernel-5.11.22-4-pve: 5.11.22-9
ceph-fuse: 15.2.14-pve1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve1
libproxmox-acme-perl: 1.4.2
libproxmox-backup-qemu0: 1.3.1-1
libpve-access-control: 7.2-4
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.2-3
libpve-guest-common-perl: 4.1-4
libpve-http-server-perl: 4.1-4
libpve-storage-perl: 7.2-10
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.0-3
lxcfs: 4.0.12-pve1
novnc-pve: 1.3.0-3
proxmox-backup-client: 2.2.7-1
proxmox-backup-file-restore: 2.2.7-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.5.1
pve-cluster: 7.2-2
pve-container: 4.2-3
pve-docs: 7.2-2
pve-edk2-firmware: 3.20220526-1
pve-firewall: 4.2-6
pve-firmware: 3.5-6
pve-ha-manager: 3.4.0
pve-i18n: 2.7-2
pve-qemu-kvm: 7.0.0-4
pve-xtermjs: 4.16.0-1
qemu-server: 7.2-4
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.7.1~bpo11+1
vncterm: 1.7-1
zfsutils-linux: 2.1.6-pve1
Code: 
   93  apt update
   94  apt dist-upgrade
   95  reboot
   96  pveversion -v
   97  history
Code: 
root@pve:~# proxmox-boot-tool status
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
System currently booted with uefi
2E85-8F0E is configured with: uefi (versions: 5.11.22-4-pve, 5.11.22-7-pve, 5.15.64-1-pve)
2E86-0B2B is configured with: uefi (versions: 5.11.22-4-pve, 5.11.22-7-pve, 5.15.64-1-pve)
Code: 
root@pve:~# cat /proc/cmdline
initrd=\EFI\proxmox\5.11.22-7-pve\initrd.img-5.11.22-7-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs
 
Last edited:
For some basic troubleshooting I added "acpi=off" to /etc/kernel/cmdline as below
Code: 
root=ZFS=rpool/ROOT/pve-1 boot=zfs acpi=off

Refreshed with proxmox-boot-tool refresh and rebooted.

Now kernel 5.15.64 boots, but but no ACPI is unacceptable. What are my next steps?

Code: 
root@pve:~# cat /proc/cmdline
initrd=\EFI\proxmox\5.15.64-1-pve\initrd.img-5.15.64-1-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs acpi=off

Code: 
root@pve:~# proxmox-boot-tool status
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
System currently booted with uefi
2E85-8F0E is configured with: uefi (versions: 5.11.22-7-pve, 5.15.64-1-pve)
2E86-0B2B is configured with: uefi (versions: 5.11.22-7-pve, 5.15.64-1-pve)

Code: 
root@pve:~# pveversion -v
proxmox-ve: 7.2-1 (running kernel: 5.15.64-1-pve)
pve-manager: 7.2-11 (running version: 7.2-11/b76d3178)
pve-kernel-5.15: 7.2-13
pve-kernel-helper: 7.2-13
pve-kernel-5.11: 7.0-10
pve-kernel-5.15.64-1-pve: 5.15.64-1
pve-kernel-5.11.22-7-pve: 5.11.22-12
pve-kernel-5.11.22-4-pve: 5.11.22-9
ceph-fuse: 15.2.14-pve1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve1
libproxmox-acme-perl: 1.4.2
libproxmox-backup-qemu0: 1.3.1-1
libpve-access-control: 7.2-4
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.2-3
libpve-guest-common-perl: 4.1-4
libpve-http-server-perl: 4.1-4
libpve-storage-perl: 7.2-10
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.0-3
lxcfs: 4.0.12-pve1
novnc-pve: 1.3.0-3
proxmox-backup-client: 2.2.7-1
proxmox-backup-file-restore: 2.2.7-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.5.1
pve-cluster: 7.2-2
pve-container: 4.2-3
pve-docs: 7.2-2
pve-edk2-firmware: 3.20220526-1
pve-firewall: 4.2-6
pve-firmware: 3.5-6
pve-ha-manager: 3.4.0
pve-i18n: 2.7-2
pve-qemu-kvm: 7.0.0-4
pve-xtermjs: 4.16.0-1
qemu-server: 7.2-4
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.7.1~bpo11+1
vncterm: 1.7-1
zfsutils-linux: 2.1.6-pve1
 
Moayad said:
Hello,

The upgrade process went ok, i.e. did not appear errors during the upgrade process?

May I ask you what exactly command you used for the upgrade? It's an apt update && apt upgrade or apt update && dist-upgrade?

EDIT: Can you also provide us with the output of pveversion -v
Click to expand...

Please let me know what troubleshooting steps are next, thank you!
 
Hello,

What does the bellow commands saying:

Bash: 
update-initramfs -k all -u
proxmox-boot-tool refresh
If no error resulting from the above commands, try to reboot the server afterward.
 
>Adding the nomodeset makes no difference, still no messages are shown.

i cannot remember that i have seen a linux system crash/hang on boot at that stage

did you remove the "quiet" param from the grub commandline ?
 
Moayad said:
Hello,

What does the bellow commands saying:

Bash: 
update-initramfs -k all -u
proxmox-boot-tool refresh
If no error resulting from the above commands, try to reboot the server afterward.
Click to expand...

Thank you for the quick reply. Sorry for the delay, I've been flying between destinations and haven't had the best connectivity.

Code: 
root@pve:~# update-initramfs -k all -u
update-initramfs: Generating /boot/initrd.img-5.15.64-1-pve
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
Copying and configuring kernels on /dev/disk/by-uuid/2E85-8F0E
        Copying kernel and creating boot-entry for 5.11.22-7-pve
        Copying kernel and creating boot-entry for 5.15.64-1-pve
Copying and configuring kernels on /dev/disk/by-uuid/2E86-0B2B
        Copying kernel and creating boot-entry for 5.11.22-7-pve
        Copying kernel and creating boot-entry for 5.15.64-1-pve
update-initramfs: Generating /boot/initrd.img-5.11.22-7-pve
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
Copying and configuring kernels on /dev/disk/by-uuid/2E85-8F0E
        Copying kernel and creating boot-entry for 5.11.22-7-pve
        Copying kernel and creating boot-entry for 5.15.64-1-pve
Copying and configuring kernels on /dev/disk/by-uuid/2E86-0B2B
        Copying kernel and creating boot-entry for 5.11.22-7-pve
        Copying kernel and creating boot-entry for 5.15.64-1-pve
update-initramfs: Generating /boot/initrd.img-5.11.22-4-pve
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
Copying and configuring kernels on /dev/disk/by-uuid/2E85-8F0E
        Copying kernel and creating boot-entry for 5.11.22-7-pve
        Copying kernel and creating boot-entry for 5.15.64-1-pve
Copying and configuring kernels on /dev/disk/by-uuid/2E86-0B2B
        Copying kernel and creating boot-entry for 5.11.22-7-pve
        Copying kernel and creating boot-entry for 5.15.64-1-pve
root@pve:~# proxmox-boot-tool refresh
Running hook script 'proxmox-auto-removal'..
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
Copying and configuring kernels on /dev/disk/by-uuid/2E85-8F0E
        Copying kernel and creating boot-entry for 5.11.22-7-pve
        Copying kernel and creating boot-entry for 5.15.64-1-pve
Copying and configuring kernels on /dev/disk/by-uuid/2E86-0B2B
        Copying kernel and creating boot-entry for 5.11.22-7-pve
        Copying kernel and creating boot-entry for 5.15.64-1-pve
root@pve:~#

Screenshot attached, no success after reboot without "acpi=off"
 

Attachments

  • Screenshot_20221113_110853.png
    Screenshot_20221113_110853.png
    10.7 KB · Views: 34
RolandK said:
>did you remove the "quiet" param from the grub commandline ?
Click to expand...
Yep, this system uses UEFI Sytemd-boot and ZFS. Below, I included the /proc/cmdline I pasted from earlier. I also tried with "no quiet". There is no additional output after the screenshot from my previous post.

Code: 
root@pve:~# cat /proc/cmdline
initrd=\EFI\proxmox\5.15.64-1-pve\initrd.img-5.15.64-1-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs acpi=off
 
RolandK said:
here is another ocurrence of the issue: https://forum.proxmox.com/threads/e...ommand-line-option-system-cannot-boot.104377/
Click to expand...
Thanks! I am already reading through this thread (page 5 ish now?). Edit: Sorry, I'm on mobile, it was this thread...https://forum.proxmox.com/threads/o...15-for-proxmox-ve-7-x-available.100936/page-6

I'm at another airport right now, tomorrow I plan to test simplefb and nofb kernel options and also any newer kernels in the pve-test repo.

Also, I'm not doing anything with PCIe or GPU passthrough, but I did try with the following iommu options, but no dice.
Code: 
intel_iommu=on
iommu=pt

I'll keep reading through that thread and see if inspiration strikes.

Any other suggestions are welcome! I'll add them to the pile for tomorrow's testing.
 
Last edited:
radim.smehlik said:
I had the same problem on my A2SDi-8C+-HLN4F after upgrading BIOS to the latest version 1.7. Boot process stucked at Loading initial ramdisk...

After downgrading BIOS to the previous version, PVE booted normally.
Click to expand...

Thank you Radim, that definitely resolved the initrd boot issue!

I was able to successfully boot kernel 5.15.64-1-pve after downgrading to 1.6.

BIOS downgraded to 1.6 using A2SDICH1.A27.zip

Pointer link: https://www.supermicro.com/support/resources/getfile.php?SoftwareItemID=15374
Final link: https://www.supermicro.com/Bios/softfiles/15374/A2SDICH1.A27.zip
File: A2SDICH1.A27.zip
SHA256: 4232c1d5eaec4d7a3805496e6fe75781bee87af026378930fd98155a351f5bfa

Credit to Serve The Home forum user RolloZ170 for the BIOS file link above.

Here is the source for that post: https://forums.servethehome.com/ind...problems-after-bios-upgrade.37706/post-350548

I tried uploading the file to the forum here (in case Supermicro ever became unavailable), but the file was too large at 5923311 bytes.

I also tried repacking with tar/zstd (--ultra -22), but only got it down to 4952197 bytes, which was still too large.

Note: I'll try additional kernel options at a later point on BIOS 1.7 to see if anything resolves the issue on that version and report back.

Thank you all for the help!
 
  • Like
Reactions: Fisxko and Neobin
Dear all,
I am sorry for not replying earlier, I have been on some business trips.

Thank you for this solution Nitokato and Radim, much appreciated!

Is this 1.6 BIOS that is posted here also suitable for my 16 core Atom C3955 motherboard A2SDI-16C-HLN4F ?
 
This worked for me too. Thanks a million!

Does anyone know if the root cause being investigated by Proxmox?
 
I experienced the same problems on my a2SDI-4C with BIOS V1.7.
The current BIOS V1.7a by supermicro solved this issue for my board. Supermicro released V1.7a very fast after the V1.7 release, so I guess the issue was caused by a faulty V1.7
 
Last edited:
  • Like
Reactions: kriegalex
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back