[SOLVED] Updating Everything Except the Kernel

[Arszilla]

Hi there,

I have a HP ProLiant DL360p Gen 8 server, running Proxmox 7.1-7. The reason I am running 7.1-7 is the known issue of my server having issues especially, in PCI Passthrough, with the 7.2 update. Additionally, I am running a custom kernel (relax-intel-rmrr) to allow PCI passthrough. Thus, I am hesitating on updating my server, but I am unable to spin up a Ubuntu 22.XY CT otherwise.

My current pve-version -v is as follows:

# pveversion -v
proxmox-ve: 7.1-1 (running kernel: 5.13.19-2-pve-relaxablermrr)
pve-manager: 7.1-7 (running version: 7.1-7/df5740ad)
pve-kernel-helper: 7.1-6
pve-kernel-5.13: 7.1-5
pve-kernel-5.13.19-2-pve: 5.13.19-4
pve-kernel-5.13.19-2-pve-relaxablermrr: 5.13.19-4
ceph-fuse: 15.2.15-pve1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve2
libproxmox-acme-perl: 1.4.0
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.1-5
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.0-14
libpve-guest-common-perl: 4.0-3
libpve-http-server-perl: 4.0-4
libpve-storage-perl: 7.0-15
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.11-1
lxcfs: 4.0.11-pve1
novnc-pve: 1.2.0-3
proxmox-backup-client: 2.1.2-1
proxmox-backup-file-restore: 2.1.2-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.4-4
pve-cluster: 7.1-2
pve-container: 4.1-2
pve-docs: 7.1-2
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: 3.3-3
pve-ha-manager: 3.3-1
pve-i18n: 2.6-2
pve-qemu-kvm: 6.1.0-3
pve-xtermjs: 4.12.0-1
qemu-server: 7.1-4
smartmontools: 7.2-1
spiceterm: 3.2-2
swtpm: 0.7.0~rc1+2
vncterm: 1.7-1
zfsutils-linux: 2.1.1-pve3

How can I update my Proxmox without ruining my current setup/configuration, especially for my custom kernel? I initially locked the kernel via
echo "pve-kernel-5.13 hold" | dpkg --set-selections but I am unsure if this will hold or not break my server if I do apt upgrade -y.

How should I proceed with updating my Proxmox instance with such a circumstance?

 

[aaron]

Check out theproxmox-boot-tool which allows you to pin kernels:

proxmox-boot-tool kernel list
proxmox-boot-tool kernel pin <kernel>

This way, it should always boot the pinned kernel, no matter if newer kernels are installed.

I am not sure though, if it is able to detect the custom kernel. Give it a try by running the kernel list command though.

 

[Arszilla]

Check out theproxmox-boot-tool which allows you to pin kernels:

proxmox-boot-tool kernel list
proxmox-boot-tool kernel pin <kernel>

This way, it should always boot the pinned kernel, no matter if newer kernels are installed.

I am not sure though, if it is able to detect the custom kernel. Give it a try by running the kernel list command though.

# proxmox-boot-tool kernel list
Manually selected kernels:
None

Automatically selected kernels:
5.13.19-2-pve
5.13.19-2-pve-relaxablermrr

However, the command is not proxmox-boot-tool kernel pin, but rather proxmox-boot-tool kernel add. Just FYI.

I assume after this and the dpkg --set-selections, I can just run apt update && apt upgrade -y without worrying about individual packages?

 

[aaron]

However, the command is not proxmox-boot-tool kernel pin, but rather proxmox-boot-tool kernel add[ICODE]. Just FYI.

After adding your kernel, you still need to pin it to make it the default kernel used at boot, not matter if newer kernels are installed.
But I did check your output of pveversion and the currently installed version of pve-kernel-helper is too old and does not yet have the "pin" feature. You need 7.1-13 or newer for that :-/

So you could also install updates and select your own kernel on the next boot manually and then use the kernel pinning to make sure that for upcoming boots, your own kernel is always used.

 

[Arszilla]

After adding your kernel, you still need to pin it to make it the default kernel used at boot, not matter if newer kernels are installed.
But I did check your output of pveversion and the currently installed version of pve-kernel-helper is too old and does not yet have the "pin" feature. You need 7.1-13 or newer for that :-/

So you could also install updates and select your own kernel on the next boot manually and then use the kernel pinning to make sure that for upcoming boots, your own kernel is always used.

The issue with the next boot is that I will possibly not have networking and broken VMs if the kernel gets replaced. I'll be trying this later tonight and cross my fingers. I'll update the post once I attempt this.

 

[Arszilla]

Remotely ran the following commands:

# apt install pve-kernel-helper --only-upgrade
# proxmox-boot-tool kernel pin 5.13.19-2-pve-relaxablermrr

By the looks of it, the kernel is now pinned as well. I will reboot the server once I am by the server in case I need to physically monitor the reboot process.

 

[Arszilla]

While performing apt upgrade -y, I noticed a newer kernel is being installed as well as my Proxmox 7.1-7 being upgraded to 7.3-1:

# apt upgrade -y
[...]

Unpacking pve-kernel-5.15 (7.3-1) ...
Preparing to unpack .../111-proxmox-ve_7.3-1_all.deb ...

Setting up pve-kernel-5.15.83-1-pve (5.15.83-1) ...
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 5.15.83-1-pve /boot/vmlinuz-5.15.83-1-pve
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 5.15.83-1-pve /boot/vmlinuz-5.15.83-1-pve

[...]

Hopefully, all will be normal upon reboot.

 

[Arszilla]

Looks like despite what it installed, it was a success @aaron!

Thanks for your help!

 

[aaron]

yeah, the kernel pinning is quite a nice feature. Once you build a new one, you can also use the "--next-boot" flag with the pinning. That should only ping it to that kernel on the next boot.

 

[Arszilla]

yeah, the kernel pinning is quite a nice feature. Once you build a new one, you can also use the "--next-boot" flag with the pinning. That should only ping it to that kernel on the next boot.

I doubt I will be able to switch to a new patched kernel, as that package hasn't been updated in a while. Worst case, I'll have to monitor my server for vulnerabilities and perform mitigations.