Opt-in Linux Kernel 5.15 for Proxmox VE 7.x available

[leesteken]

This work-around might be of interest for people who use vendor-reset with kernel 5.15 (or later).

AFTER loading the module (after dmesg says vendor_reset_hook: installed), run echo 'device_specific' > /sys/bus/pci/devices/<pci_device_id_here>/reset_method

 

[entilza]

Waiting for 7.2 proxmox with it all integrated. Any idea when it will be out?

 

[t.lamprecht]

Waiting for 7.2 proxmox with it all integrated. Any idea when it will be out?

I mean, there's a hint in my opening post of this thread

The next Proxmox VE point release 7.2 (~ Q2/2022) will presumably use the 5.15 based Linux kernel.

 

[entilza]

Thank you Thomas

 

[atari]

Im runing:
Linux pve 5.15.5-1-pve #1 SMP PVE 5.15.5-1 (Mon, 29 Nov 2021 18:49:57 +0100) x86_64 GNU/Linux
proxmox-ve: 7.1-1 (running kernel: 5.15.5-1-pve)
pve-manager: 7.1-8 (running version: 7.1-8/5b267f33)
pve-kernel-5.15: 7.1-7
pve-kernel-helper: 7.1-6
pve-kernel-5.13: 7.1-5
pve-kernel-5.15.7-1-pve: 5.15.7-1
pve-kernel-5.15.5-1-pve: 5.15.5-1
pve-kernel-5.13.19-2-pve: 5.13.19-4
ceph-fuse: 15.2.14-pve1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve2
libproxmox-acme-perl: 1.4.0
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.1-5
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.0-14
libpve-guest-common-perl: 4.0-3
libpve-http-server-perl: 4.0-4
libpve-storage-perl: 7.0-15
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.11-1
lxcfs: 4.0.11-pve1
novnc-pve: 1.3.0-1
proxmox-backup-client: 2.1.2-1
proxmox-backup-file-restore: 2.1.2-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.4-4
pve-cluster: 7.1-3
pve-container: 4.1-3
pve-docs: 7.1-2
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: 3.3-4
pve-ha-manager: 3.3-1
pve-i18n: 2.6-2
pve-qemu-kvm: 6.1.0-3
pve-xtermjs: 4.12.0-1
qemu-server: 7.1-4
smartmontools: 7.2-1
spiceterm: 3.2-2
swtpm: 0.7.0~rc1+2
vncterm: 1.7-1
zfsutils-linux: 2.1.1-pve3
& install https://github.com/Ta180m/zenpower3
but still only get:
watch -n.5 "sensors 'zenpower-*'"
zenpower-pci-00c3
Adapter: PCI adapter
Tdie: +77.1°C (high = +95.0°C)
Tctl: +77.1°C

./zp_read_debug.sh
KERN_SUP: 1
NODE0; CPU0; N/CPU: 1
0005a008 = 00000002
0005a00c = 015c0007
0005a010 = 012f0051
0005a014 = 00000000
000598bc = 0fff0fff
0005994c = 00000000
00059954 = 00000bcc
00059958 = 00000bdc
0005995c = 00000000
00059960 = 00000000
00059964 = 00000000
00059968 = 00000000
0005996c = 00000000
00059970 = 00000000

 

[atari]

Linux pve 5.15.7-1-pve #1 SMP PVE 5.15.7-1 (Tue, 14 Dec 2021 16:42:34 +0100) x86_64 GNU/Linux
proxmox-ve: 7.1-1(running kernel: 5.15.7-1-pve)
watch -n.5 "sensors 'zenpower-*'"
zenpower-pci-00c3
Adapter: PCI adapter
SVI2_Core: 1.48 V
SVI2_SoC: 975.00 mV
Tdie: +37.1°C (high = +95.0°C)
Tctl: +37.1°C
Tccd1: +44.2°C
Tccd2: +42.0°C
SVI2_P_Core: 30.27 W
SVI2_P_SoC: 2.01 W
SVI2_C_Core: 20.42 A
SVI2_C_SoC: 2.06 A

 

[birdy]

Almost 24h on the way and all works ok. Only noticable is the much higher CPU usage.

Most of my VMs do not drop below 20% anymore where they were at approx. 4-5% on average before.

This was the solution:

https://forum.proxmox.com/threads/inconsistencies-on-cpu-usage-inside-guest-and-from-kvm.63053/

 

[coolspot]

Works great on my Ryzen 7 5800X w/ Gigabyte X570 UD rev 1.1 motherboard and Timetec DDR4 RAM. It also seems to have solved my machine check exceptions (MCE), I have not seen one in >24 hours.

 

[amplisys]

I've been having performance issues since upgrading from pve 6.4 to pve 7.2 on HP DL380 G9 with 8x consumer grade SSDs. Upgrading to 5.15.5 (opt-in) kernel has 2.7x my fsyncs, while moving to 5.15.7 has 2.5x fsyncs.

root@dc2mox014:~# pveperf
CPU BOGOMIPS:      239894.88
REGEX/SECOND:      2686054
HD SIZE:           3596.43 GB (rpool/ROOT/pve-1)
FSYNCS/SECOND:     290.69
DNS EXT:           41.10 ms
DNS INT:           39.02 ms (gtlpn.com)

root@dc2mox014:~# uname -a
Linux dc2mox014 5.13.19-2-pve #1 SMP PVE 5.13.19-4 (Mon, 29 Nov 2021 12:10:09 +0100) x86_64 GNU/Linux

----

root@dc2mox014:~# pveperf
CPU BOGOMIPS:      239877.36
REGEX/SECOND:      2828698
HD SIZE:           3596.44 GB (rpool/ROOT/pve-1)
FSYNCS/SECOND:     786.04
DNS EXT:           31.98 ms
DNS INT:           18.91 ms (gtlpn.com)

root@dc2mox014:~# uname -a
Linux dc2mox014 5.15.5-1-pve #1 SMP PVE 5.15.5-1 (Mon, 29 Nov 2021 18:49:57 +0100) x86_64 GNU/Linux

----

root@dc2mox014:~# pveperf
CPU BOGOMIPS:      239775.60
REGEX/SECOND:      2805161
HD SIZE:           3596.44 GB (rpool/ROOT/pve-1)
FSYNCS/SECOND:     733.77
DNS EXT:           28.12 ms
DNS INT:           16.39 ms (gtlpn.com)

root@dc2mox014:~# uname -a
Linux dc2mox014 5.15.7-1-pve #1 SMP PVE 5.15.7-1 (Tue, 14 Dec 2021 16:42:34 +0100) x86_64 GNU/Linux

 

[mijanek]

Great build. Also nested virtualisation is working fine on mine Ryzen 7 4800U with Windows 2022. (Was not starting on previus 5.13 kernel)
@proxmox staff: Big Thanks

 

[tlex]

Please excuse my low level question but I'm still a newbie with linux...
I've updated the kernel running :

apt update && apt install pve-kernel-5.15

after a reboot I see that I'm running the correct kernel :

pveversion
pve-manager/7.1-10/6ddebafe (running kernel: 5.15.12-1-pve)

But now I have issues running regular updates :

apt upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
4 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] y
Setting up pve-kernel-5.13.19-3-pve (5.13.19-6) ...
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 5.13.19-3-pve /boot/vmlinuz-5.13.19-3-pve
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 5.13.19-3-pve /boot/vmlinuz-5.13.19-3-pve
update-initramfs: Generating /boot/initrd.img-5.13.19-3-pve
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
Copying and configuring kernels on /dev/disk/by-uuid/C6D5-A6CE
    Copying kernel and creating boot-entry for 5.13.19-3-pve
    Copying kernel and creating boot-entry for 5.15.12-1-pve
    Copying kernel and creating boot-entry for 5.15.7-1-pve
Copying and configuring kernels on /dev/disk/by-uuid/C6D5-D464
    Copying kernel and creating boot-entry for 5.13.19-3-pve
    Copying kernel and creating boot-entry for 5.15.12-1-pve
    Copying kernel and creating boot-entry for 5.15.7-1-pve
run-parts: executing /etc/kernel/postinst.d/kdump-tools 5.13.19-3-pve /boot/vmlinuz-5.13.19-3-pve
kdump-tools: Generating /var/lib/kdump/initrd.img-5.13.19-3-pve
mkinitramfs: failed to determine device for /
mkinitramfs: workaround is MODULES=most, check:
grep -r MODULES /var/lib/kdump/initramfs-tools

Error please report bug on initramfs-tools
Include the output of 'mount' and 'cat /proc/mounts'
update-initramfs: failed for /var/lib/kdump/initrd.img-5.13.19-3-pve with 1.
run-parts: /etc/kernel/postinst.d/kdump-tools exited with return code 1
Failed to process /etc/kernel/postinst.d at /var/lib/dpkg/info/pve-kernel-5.13.19-3-pve.postinst line 19.
dpkg: error processing package pve-kernel-5.13.19-3-pve (--configure):
 installed pve-kernel-5.13.19-3-pve package post-installation script subprocess returned error exit status 2
Setting up pve-kernel-5.15.12-1-pve (5.15.12-2) ...
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 5.15.12-1-pve /boot/vmlinuz-5.15.12-1-pve
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 5.15.12-1-pve /boot/vmlinuz-5.15.12-1-pve
update-initramfs: Generating /boot/initrd.img-5.15.12-1-pve
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
Copying and configuring kernels on /dev/disk/by-uuid/C6D5-A6CE
    Copying kernel and creating boot-entry for 5.13.19-3-pve
    Copying kernel and creating boot-entry for 5.15.12-1-pve
    Copying kernel and creating boot-entry for 5.15.7-1-pve
Copying and configuring kernels on /dev/disk/by-uuid/C6D5-D464
    Copying kernel and creating boot-entry for 5.13.19-3-pve
    Copying kernel and creating boot-entry for 5.15.12-1-pve
    Copying kernel and creating boot-entry for 5.15.7-1-pve
run-parts: executing /etc/kernel/postinst.d/kdump-tools 5.15.12-1-pve /boot/vmlinuz-5.15.12-1-pve
kdump-tools: Generating /var/lib/kdump/initrd.img-5.15.12-1-pve
mkinitramfs: failed to determine device for /
mkinitramfs: workaround is MODULES=most, check:
grep -r MODULES /var/lib/kdump/initramfs-tools

Error please report bug on initramfs-tools
Include the output of 'mount' and 'cat /proc/mounts'
update-initramfs: failed for /var/lib/kdump/initrd.img-5.15.12-1-pve with 1.
run-parts: /etc/kernel/postinst.d/kdump-tools exited with return code 1
Failed to process /etc/kernel/postinst.d at /var/lib/dpkg/info/pve-kernel-5.15.12-1-pve.postinst line 19.
dpkg: error processing package pve-kernel-5.15.12-1-pve (--configure):
 installed pve-kernel-5.15.12-1-pve package post-installation script subprocess returned error exit status 2
dpkg: dependency problems prevent configuration of pve-kernel-5.13:
 pve-kernel-5.13 depends on pve-kernel-5.13.19-3-pve; however:
  Package pve-kernel-5.13.19-3-pve is not configured yet.

dpkg: error processing package pve-kernel-5.13 (--configure):
 dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of pve-kernel-5.15:
 pve-kernel-5.15 depends on pve-kernel-5.15.12-1-pve; however:
  Package pve-kernel-5.15.12-1-pve is not configured yet.

dpkg: error processing package pve-kernel-5.15 (--configure):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 pve-kernel-5.13.19-3-pve
 pve-kernel-5.15.12-1-pve
 pve-kernel-5.13
 pve-kernel-5.15
E: Sub-process /usr/bin/dpkg returned an error code (1)

I know this must be trivial but I can't figure... anyone can assist me with that ?

 

[birdy]

You shouldn’t use apt upgrade, only apt full-upgrade or apt dist-upgrade

 

[tlex]

You shouldn’t use apt upgrade, only apt full-upgrade or apt dist-upgrade

I'm having the exact same results with

apt dist-upgrade

and

apt full-upgrade

 

[t.lamprecht]

4 not fully installed or removed.

The package manager is not in a clean state and that has highly probably nothing to do with the installation of the pve-kernel-5.15 package itself. Can you please open a new thread about this?

 

[tlex]

The package manager is not in a clean state and that has highly probably nothing to do with the installation of the pve-kernel-5.15 package itself. Can you please open a new thread about this?

I did : https://forum.proxmox.com/threads/problem-after-updating-kernel.103293/

But I really think this happened after I opt-in for the latest kernel :

dpkg -C
The following packages have been unpacked but not yet configured.
They must be configured using dpkg --configure or the configure
menu option in dselect for them to work:
 pve-kernel-5.13      Latest Proxmox VE Kernel Image
 pve-kernel-5.15      Latest Proxmox VE Kernel Image

The following packages are only half configured, probably due to problems
configuring them the first time.  The configuration should be retried using
dpkg --configure <package> or the configure menu option in dselect:
 pve-kernel-5.13.19-3-pve The Proxmox PVE Kernel Image
 pve-kernel-5.15.12-1-pve The Proxmox PVE Kernel Image

 

[Oggy512]

Intel I219-V (still or again) not working:

Syslog during boot says:

e1000e 0000:00:1f.6: The NVM Checksum Is Not Valid
e1000e: probe of 0000:00:1f.6 failed with error -5

proxmox-ve: 7.1-1 (running kernel: 5.15.12-1-pve)
pve-manager: 7.1-8 (running version: 7.1-8/5b267f33)
pve-kernel-5.15: 7.1-8
pve-kernel-helper: 7.1-6
pve-kernel-5.13: 7.1-5
pve-kernel-5.15.12-1-pve: 5.15.12-2
pve-kernel-5.13.19-2-pve: 5.13.19-4
pve-kernel-5.11.22-4-pve: 5.11.22-9
ceph-fuse: 15.2.14-pve1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve2
libproxmox-acme-perl: 1.4.0
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.1-5
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.0-14
libpve-guest-common-perl: 4.0-3
libpve-http-server-perl: 4.0-4
libpve-storage-perl: 7.0-15
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.11-1
lxcfs: 4.0.11-pve1
novnc-pve: 1.3.0-1
proxmox-backup-client: 2.1.2-1
proxmox-backup-file-restore: 2.1.2-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.4-4
pve-cluster: 7.1-3
pve-container: 4.1-3
pve-docs: 7.1-2
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: 3.3-4
pve-ha-manager: 3.3-1
pve-i18n: 2.6-2
pve-qemu-kvm: 6.1.0-3
pve-xtermjs: 4.12.0-1
qemu-server: 7.1-4
smartmontools: 7.2-1
spiceterm: 3.2-2
swtpm: 0.7.0~rc1+2
vncterm: 1.7-1
zfsutils-linux: 2.1.1-pve3

Same error occurs with kenel 5.13, however 5.11.22-4 is working fine.
See also my thread here

 

[kromberg]

Updates today from "deb http://download.proxmox.com/debian bullseye pve-no-subscription" broke the management port. Can not ssh to it or bring up the UI from it. Using ssh it keeps failing the password. Pointing a browser to port 8006 gets connection refused. Any chance of getting a fix for this soon? Guessing the package that broke things is pve-manager 7.1-10.

 

[headwhacker]

I have 2 PVE cluster both running the latest 7.1-10 package. UI/ssh still working fine.

 

[t.lamprecht]

Updates today from "deb http://download.proxmox.com/debian bullseye pve-no-subscription" broke the management port. Can not ssh to it or bring up the UI from it. Using ssh it keeps failing the password. Pointing a browser to port 8006 gets connection refused. Any chance of getting a fix for this soon? Guessing the package that broke things is pve-manager 7.1-10.

This has nothing to do with the topic of this thread, the new opt-in 5.15 based kernel... Also, as others mentioned, that package does not result in such breakage on its own. Your symptoms sounds more like a network issue, i.e., on a deeper level. I'd check the sate of the system via remote (iKVM/IPMI) or physical access console, if that doesn't already show the issue I'd recommend posting a new thread here with more details about the system in general and about what was done since the last boot it worked and before it broke.

 

[blueberry]

I ran into multiple kernel oops running the 5.15 kernel and Netdata with eBPF, which took out all Ceph-I/O on the affected host.
More details can be found here: https://github.com/netdata/netdata/issues/12027

I'm not sure if this is the kernels or Netdata's fault (although the Netdata developers had issues with Ubuntu kernels in the past), but I still wanted to leave this here in case anyone else has similar problems or the Proxmox developers have any insight or ideas.