Opt-in Linux 5.19 Kernel for Proxmox VE 7.x available

t.lamprecht

Proxmox Staff Member
Staff member
Jul 28, 2015
5,504
1,753
164
South Tyrol/Italy
shop.proxmox.com
We recently uploaded a 5.19 kernel into our repositories. The 5.15 kernel will stay the default on the Proxmox VE 7.x series, 5.19 is an option.
5.19 may be useful for some (especially newer) setups, for example if there is improved hardware support that has not yet been backported to 5.15.

How to install:
  • apt update
  • apt install pve-kernel-5.19
  • reboot
Future updates to the 5.19 kernel will now get installed automatically.

Please note:
  • It's not required to enable the pvetest repository, the opt-in kernel package is available on all repositories.
  • While we are trying to provide a stable experience with the opt-in 5.19 kernel, updates may appear less frequently than for the default 5.15 based kernel.
  • If unsure, we recommend continuing to use the 5.15 based kernel.
Feedback is welcome!
 

leesteken

Famous Member
May 31, 2020
2,360
502
118
Passthrough of two AMD GPU (with vendor-reset), audio, SATA and USB controllers works well on X570S. No more need of initcall_blacklist=sysfb_init work-around for passthrough of boot GPU, which was needed after 5.11.22 (because amdgpu crashes when unloading) until 5.15.35 and after recent (Debian 11.5?) update again.
lm-sensors still does not detect it8628 on X570S AERO G but work-around still works and not related to Proxmox.
I do not notice any regressions but I also don't see the wlan device of the mt7921e (driver in use) but that probably requires kernel 5.19.8 (or 5.15.67).
 
  • Like
Reactions: psyyo and Neobin

t.lamprecht

Proxmox Staff Member
Staff member
Jul 28, 2015
5,504
1,753
164
South Tyrol/Italy
shop.proxmox.com

itNGO

Well-Known Member
Jun 12, 2020
570
125
48
44
Germany
it-ngo.com
Those are in 5.15 too since end of July, i.e. pve-kernel-5.15.39-2.
Did anyone made measurement if and how much this slows performance down in PVE?
 

fabian

Proxmox Staff Member
Staff member
Jan 7, 2016
8,137
1,587
164
Did anyone made measurement if and how much this slows performance down in PVE?
well, that depends a lot on hardware and workload, like with all of these mitigations. you can disable it via the kernel cmdline and check for yourself (and then decide which tradeoff you want to make w.r.t. performance vs. security/safety, like with all optional mitigations and security features).

note that the kernel devs are already working on re-gaining a lot of the performance lost with this specific mitigation, but the changes are not trivial so likely it will take a bit before they become available in released kernel versions (upstream and in Proxmox products): https://lore.kernel.org/all/20220902130625.217071627@infradead.org/
 

Neobin

Renowned Member
Apr 29, 2021
860
289
63
Finally the latest and greatest. :cool:
Successfully installed and running on:
  • 5950X on X570 with nvidia-driver=470.141.03
  • 5700G on X300
  • 9900K on Z390 with kde-plasma-desktop
All are completely on bullseye-backports and have PBS aside.

Only the Intel throwed some possible missing firmware warnings at me:
Bash:
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  pve-kernel-5.19.7-1-pve
The following NEW packages will be installed:
  pve-kernel-5.19 pve-kernel-5.19.7-1-pve
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 77.6 MB of archives.
After this operation, 412 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://download.proxmox.com/debian/pbs bullseye/pbs-no-subscription amd64 pve-kernel-5.19.7-1-pve amd64 5.19.7-1 [77.6 MB]
Get:2 http://download.proxmox.com/debian/pbs bullseye/pbs-no-subscription amd64 pve-kernel-5.19 all 7.2-11 [4,976 B]
Fetched 77.6 MB in 2s (31.2 MB/s)         
Selecting previously unselected package pve-kernel-5.19.7-1-pve.
(Reading database ... 182580 files and directories currently installed.)
Preparing to unpack .../pve-kernel-5.19.7-1-pve_5.19.7-1_amd64.deb ...
Unpacking pve-kernel-5.19.7-1-pve (5.19.7-1) ...
Selecting previously unselected package pve-kernel-5.19.
Preparing to unpack .../pve-kernel-5.19_7.2-11_all.deb ...
Unpacking pve-kernel-5.19 (7.2-11) ...
Setting up pve-kernel-5.19.7-1-pve (5.19.7-1) ...
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 5.19.7-1-pve /boot/vmlinuz-5.19.7-1-pve
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 5.19.7-1-pve /boot/vmlinuz-5.19.7-1-pve
update-initramfs: Generating /boot/initrd.img-5.19.7-1-pve
W: Possible missing firmware /lib/firmware/i915/dg1_huc_7.9.3.bin for module i915
W: Possible missing firmware /lib/firmware/i915/tgl_guc_69.0.3.bin for module i915
W: Possible missing firmware /lib/firmware/i915/adlp_guc_69.0.3.bin for module i915
W: Possible missing firmware /lib/firmware/i915/skl_guc_70.1.1.bin for module i915
W: Possible missing firmware /lib/firmware/i915/bxt_guc_70.1.1.bin for module i915
W: Possible missing firmware /lib/firmware/i915/kbl_guc_70.1.1.bin for module i915
W: Possible missing firmware /lib/firmware/i915/glk_guc_70.1.1.bin for module i915
W: Possible missing firmware /lib/firmware/i915/kbl_guc_70.1.1.bin for module i915
W: Possible missing firmware /lib/firmware/i915/kbl_guc_70.1.1.bin for module i915
W: Possible missing firmware /lib/firmware/i915/cml_guc_70.1.1.bin for module i915
W: Possible missing firmware /lib/firmware/i915/icl_guc_70.1.1.bin for module i915
W: Possible missing firmware /lib/firmware/i915/ehl_guc_70.1.1.bin for module i915
W: Possible missing firmware /lib/firmware/i915/ehl_guc_70.1.1.bin for module i915
W: Possible missing firmware /lib/firmware/i915/tgl_guc_70.1.1.bin for module i915
W: Possible missing firmware /lib/firmware/i915/tgl_guc_70.1.1.bin for module i915
W: Possible missing firmware /lib/firmware/i915/dg1_guc_70.1.1.bin for module i915
W: Possible missing firmware /lib/firmware/i915/tgl_guc_70.1.1.bin for module i915
W: Possible missing firmware /lib/firmware/i915/adlp_guc_70.1.1.bin for module i915
W: Possible missing firmware /lib/firmware/i915/dg2_guc_70.1.2.bin for module i915
W: Possible missing firmware /lib/firmware/i915/adlp_dmc_ver2_16.bin for module i915
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
Copying and configuring kernels on /dev/disk/by-uuid/7048-8851
        Copying kernel and creating boot-entry for 5.15.53-1-pve
        Copying kernel and creating boot-entry for 5.19.7-1-pve
        Removing old version 5.15.39-4-pve
Couldn't find EFI system partition. It is recommended to mount it to /boot or /efi.
Alternatively, use --esp-path= to specify path to mount point.
run-parts: executing /etc/kernel/postinst.d/proxmox-auto-removal 5.19.7-1-pve /boot/vmlinuz-5.19.7-1-pve
run-parts: executing /etc/kernel/postinst.d/systemd-boot 5.19.7-1-pve /boot/vmlinuz-5.19.7-1-pve
Couldn't find EFI system partition. It is recommended to mount it to /boot or /efi.
Alternatively, use --esp-path= to specify path to mount point.
run-parts: executing /etc/kernel/postinst.d/unattended-upgrades 5.19.7-1-pve /boot/vmlinuz-5.19.7-1-pve
run-parts: executing /etc/kernel/postinst.d/zz-proxmox-boot 5.19.7-1-pve /boot/vmlinuz-5.19.7-1-pve
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
Copying and configuring kernels on /dev/disk/by-uuid/7048-8851
        Copying kernel and creating boot-entry for 5.15.53-1-pve
        Copying kernel and creating boot-entry for 5.19.7-1-pve
run-parts: executing /etc/kernel/postinst.d/zz-update-grub 5.19.7-1-pve /boot/vmlinuz-5.19.7-1-pve
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.19.7-1-pve
Found initrd image: /boot/initrd.img-5.19.7-1-pve
Found linux image: /boot/vmlinuz-5.15.53-1-pve
Found initrd image: /boot/initrd.img-5.15.53-1-pve
Found linux image: /boot/vmlinuz-5.15.39-4-pve
Found initrd image: /boot/initrd.img-5.15.39-4-pve
Found memtest86+ image: /ROOT/pve-1@/boot/memtest86+.bin
Found memtest86+ multiboot image: /ROOT/pve-1@/boot/memtest86+_multiboot.bin
Warning: os-prober will not be executed to detect other bootable partitions.
Systems on them will not be added to the GRUB boot configuration.
Check GRUB_DISABLE_OS_PROBER documentation entry.
Adding boot menu entry for UEFI Firmware Settings ...
done
Setting up pve-kernel-5.19 (7.2-11) ...
 

celemine1gig

Member
May 22, 2020
11
0
6
52
Just tried the Kernel 5.19 on my Xeon E5 v4 system with passed through GTX 980 Ti under Windows.
Experienced massive lags/delays in games. Same setup runs smooth if I switch back to 5.15. I guess there still is work to do.
BTW, nothing obvious in the system log, when the delays hit. And the interrupt setup also is the same. Double-checked.
 

t.lamprecht

Proxmox Staff Member
Staff member
Jul 28, 2015
5,504
1,753
164
South Tyrol/Italy
shop.proxmox.com
Experienced massive lags/delays in games. Same setup runs smooth if I switch back to 5.15. I guess there still is work to do.
Note that with 5.19 the schedulers preemption types are finally not compiled in anymore and can be set on boot. You could try to adapt those:

Code:
 preempt=        [KNL]
                 Select preemption mode if you have CONFIG_PREEMPT_DYNAMIC
                 none - Limited to cond_resched() calls
                 voluntary - Limited to cond_resched() and might_sleep() calls
                 full - Any section that isn't explicitly preempt disabled
                        can be preempted anytime.
-- Documentation/admin-guide/kernel-parameters.txt

none means most throughput (often best for servers), voluntary brings less latency for processes (often better for desktop/ui) at the cost of some smaller throughput impact (this is somewhat of a balance), and full has the lowest latency but also the biggest impact on throughput. It seems default is "voluntary"

You could try booting with preempt=none to see if that improves your situation. Nnote that with VMs the host OS giving more throughput to the guest OS should mean that the guest OS has more control about its guest perceived latency. Would be interesting to get feedback here.
 

celemine1gig

Member
May 22, 2020
11
0
6
52
Tried all three kernel parameters with the current 5.19 pve-kernel. Sadly none brought back the previous functionality, as seen on 5.15.
I do not have measurements, but it felt like "voluntary" was the worst, followed by "full" and the most acceptable (yet not nearly as good as on 5.15) was "none".
 

celemine1gig

Member
May 22, 2020
11
0
6
52
BTW, another quick question, about the upcoming kernel 5.19:
Is there a chance to solve that dreadful simplefb dilemma with the 5.19 configuration, concerning simplefb.
I have a Proxmox CoffeeLake system with passed-though iGPU. Up until 5.13 this worked (and still works) beautifully.
However, since 5.15, the simplefb claims the iGPU and, in contrast to external PCIe devices, the workaround by resetting the device, does not work.
For the internal function, this seems more or less like an unsolveable situation, as long as simplefb claims those devices by default.
 

leesteken

Famous Member
May 31, 2020
2,360
502
118
However, since 5.15, the simplefb claims the iGPU and, in contrast to external PCIe devices, the workaround by resetting the device, does not work.
The work-around with the initcall_blacklist=sysfb_init kernel parameter also does not work for you?
 

celemine1gig

Member
May 22, 2020
11
0
6
52
OK, so you can scratch my request, as I obviously did not know about that option.
Just tested and works fine with 5.15 and 5.19.
Now that you mentioned it, I think I remember having read about it some time ago, but then forgot to try it out.
Thanks anyway! Much appreciated input.
 

Dark26

Active Member
Nov 27, 2017
233
17
38
45
I try to install it on my N5105 server , and it crash after booting with this version of 5.19
 

Dark26

Active Member
Nov 27, 2017
233
17
38
45
What crashes (host, VM, ..?) and do you got any specifc error logs that show up before/during crash? Also more details about the HW could be nice to have.

With the 5.15, host stable, and one vm ( debian with container ( bitwarder ) crash. No problem with oher Vm / lxc.

with 5.19, the host boot, but it seem that when it start lauching the VM /lxc , the hosts crash.

i watch quickly, the log, and maybe it's a storage problem ( Nfs ganesha over glusterfs not starting automaticly a boot) and when the VM start on the storage, everything crash.

i will watch to reboot with the secreen on when i have time
 
  • Like
Reactions: t.lamprecht

Randell

Member
Dec 27, 2018
15
4
8
44
It looks like it has resolved my migration issues to/from an i7-12700K and i7-8700K machine.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!