Installing pve-headers package causes PVE to fail to boot due to initramfs out of memory error

kayson

New Member
Feb 13, 2024
27
1
3
On a fresh PVE 8.3-1 install, I add the no subscription repository, and install pve-headers. It looks like it also installs the 6.8.12-8 kernel and updates initramfs. After that, the installation is toast and unrecoverable because it compalins of out of memory. Even booting the original kernel version is broken.

How can I install pve-headers without the updated kernel? Or how can I successfully update the kernel?
 
Last edited:
Hello kayson! If you go to "Advanced options ..." at boot, are you able to boot to recovery mode?
No. Both kernels, recovery mode or not, fail to start. Probably because, from what I remember, recovery mode just changes kernel options, but something has broken the initramfs which gets loaded first.
 
Have you done anything else except installing the pve-headers / proxmox-headers? Also, are you able to boot any other kernel version in any mode at all? I'm just trying to understand the situation, because only installing the pve-headers / proxmox-headers alone should not lead to such issues.

I had a similar issue some time ago with my own computer. In my case, I did the following:
  1. Adapted /etc/initramfs-tools/initramfs.confto use the following values:
    • COMPRESS=xz
    • COMPRESSLEVEL=9
    • I just want to mention that using either of the previous values would have probably worked as well, but I just wanted to be sure and enable the maximum compression.
  2. Execute update-initramfs -u -k all

Of course, if you have issues booting with all kernels, you will need to use a separate live distro, chroot into the Proxmox VE installation and try what I suggested above.

Either way, even if you manage to fix it, it would be great to let us know what you did before the issue happened.
 
Literally nothing else. Fresh PVE install, add the no subscription repo, install the package, and reboot. I'm using an automated install with the below toml file. It looks like the automated installer iso uses a kernel param ramdisk_size=16777216 which then makes it into the installed grub cfg. Maybe that's not big enough for whatever changes the package is making to initramfs?

I did already try repairing initramfs with a live distro but didn't have any luck.

Code:
[global]keyboard = "en-us"
country = "us"
fqdn = "pve.REDACTED"
mailto = "REDACTED"
timezone = "America/Los_Angeles"
root_password = "REDACTED"
root_ssh_keys = []
reboot_on_error = false

[network]
source = "from-dhcp"

[disk-setup]
filesystem = "ext4"
filter.ID_SERIAL = "BC711*"
lvm.hdsize = 50
lvm.swapsize = 0
lvm.maxvz = 0
 
Sorry to hear that. Just to be sure: have you already tried compressing it further, like I suggested above?
I had time to do some more digging. On a fresh PVE 8.3-1 install from the automated iso, the initrd image is 59MB. initramfs.conf appears to be set to us zstd compression at level 9. I was wrong about the ramdisk_size option. That's just for the installer run; it doesn't get saved to the OS disk.

Then I update the apt source to use the no subscription repository and run apt install pve-headers. This also installs the 6.8.12-9 kernel now. After that, I have a new initrd for the new kernel. It is also 59MB, so I don't think the size actually has anything to do with it. After a reboot, whether I boot into 6.8.12-9 or 6.8.12-4, recovery mode or not, the boot is unsuccessful because of the error mentioned above.

I started up a debian live session and updated the initrd's with xz level 9. File size for both goes down to 48MB.

Still get the same error. Even doing recovery mode and trying a very very basic kernel command-line, etc, does not work. Not sure what exactly is happening, but I don't think it's actually initrd
 
Ok I tracked it down. It's related to Intel SGX. If I disable it in the BIOS, then I can boot just fine. I'm not sure why the kernel update is causing this problem... any ideas?