Kernel 5.11

[martin]

We just uploaded a 5.11 kernel into our repositories. The 5.4 kernel is still the default on Proxmox VE 6.x series, 5.11 is an option.

How to install:

• apt update
• apt install pve-kernel-5.11
• reboot

Future updates to the 5.11 kernel will now get installed automatically.

Feedback is welcome!

 

[Ramalama]

Omg

Only for those that used "pve-edge" kernels before:
- please revert back /etc/apparmor/parser.conf
-> features-file=/usr/share/apparmor-features/features.stock (Doesn't work here)
-> features-file=/usr/share/apparmor-features/features (Is correct & works!)

Feedback:
So far everything looks great! Thanks

 

[Ramalama]

Okay, first issue and i don't know if this is really one, because i tested myself 5.11.7 very short, before this official release.
- RDRAND again xD
- I don't know why, but in the last week on my own 5.11.5 compiled kernel, 10 of 10 boots were succesfull, not a single Service failed during boot.
- With this kernel here, 2 out of 5 boots, 2 Services failed.
I don't want to exclude that it's 5.11.7 itself or that i simply had luck on my kernel 10/10 or is it probably gcc-11? Dunno if gcc-11 vs 10 can cause such things. My kernel was compiled with -o3 -march=znver3 and gcc-11. But i doubt very high that this has anything with that todo.

However, im going to compile 5.11.7 on my side and will reboot some times xD
If this issue persists, it's still 10000% better as with 5.4.

For everyone else, this happens only on Zen3 (Agesa <1.2.0.0). So it is very specific. (Agesa 1.2.0.0 fixes this)

Cheers

 

[Ramalama]

@t.lamprecht

git clone git://git.proxmox.com/git/pve-kernel.git
cd pve-kernel
- git submodule update --init --depth=1 --recursive submodules/ubuntu-hirsute
- git submodule update --init --recursive
+ make submodule

Did i missed something?

Because:

debian/scripts/abi-generate debian/pve-headers-5.11.7-1-pve/usr/src/linux-headers-5.11.7-1-pve/Module.symvers abi-5.11.7-1-pve 5.11.7-1-pve
input file 'debian/pve-headers-5.11.7-1-pve/usr/src/linux-headers-5.11.7-1-pve/Module.symvers' does not exist

Because i don't have this src folder: pve-kernel/build/debian/pve-headers-5.11.7-1-pve/usr/src/

EDIT:
Dunno what the error was, maybe checked out in the wrong time yesterday?
However, tryed today, compiled successfull, testing now. Thanks!

Update:
1. It's the first kernel that i can compile without errors with -flto... But the resulting kernel has 13kb, so im not going to try that xD
2. After compilation (without lto), i have tryed it and well, same rdrand, but extremely rare 1 out of 5 boots, 2 services.
Same with the old 5.11.5, seems like i had simply last week a good run xD
However it's working well, thanks for the updated kernel guys, great work! Especially thanks to lamprecht, i think he maintains it.

Dunno why other people have so many issues, but here with (Ryzen 5800x + ecc + asrock x570d4i + zfs + nvme + kernel 5.11) proxmox runs rock solid & super fast! No issues with passthrough or anything at all... And i have tested a lot.
And no weird freezing issues after time or backup tasks freezing or whatever people have issues with. (Except if rdrand hits rarely at rebooting)

Thanks & Cheers!

 

[j1a2o]

Is this planning to get promoted to pve-no-subscription at some point?

 

[filipealvarez]

What are the news features/benefits of this kernel? Thanks

 

[tom]

Is this planning to get promoted to pve-no-subscription at some point?

Already there.

 

[ness1602]

Hi tom, i can confirm it is working, even with Epyc servers.

 

[t.lamprecht]

What are the news features/benefits of this kernel? Thanks

Basically: It's newer, so more features, more/better hardware support but less battle-tested in general.
Hardware support is still backported to the 5.4 kernel, but it isn't always straight forward to do so, so newer kernel major version will normally still provide better support for relatively new HW.

If you want an in-depth, but still "human-readable", review of the changes since the 5.4 kernel I suggest checking out the Kernel Newbies changelog pages, they are quite good IMO:

https://kernelnewbies.org/Linux_5.5
https://kernelnewbies.org/Linux_5.6
https://kernelnewbies.org/Linux_5.7
https://kernelnewbies.org/Linux_5.8
https://kernelnewbies.org/Linux_5.9
https://kernelnewbies.org/Linux_5.10
https://kernelnewbies.org/Linux_5.11

 

[comfreak]

Hi,

its working so far on my enviroment (AMD Ryzen Threadripper 1900X).

I wanted to test the btrfs raid1c3 feature, which was released on Kernel Version 5.5 but the profile is still missing?

https://kernelnewbies.org/Linux_5.5#Btrfs_RAID1_with_3_and_4_copies_and_more_checksum_alternatives

root@proxmoxnested:~# mkfs.btrfs -L data -d raid6 -m raid1c3 -f /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf
ERROR: unknown profile raid1c3
root@proxmoxnested:~# uname -r
5.11.7-1-pve

Also the manpage says it is not available?

 

[marcosscriven]

What’s actually in the Proxmox-specific kernel? So far as I can tell from the repo it’s just a few patches like ACS.

if we don’t need those patches, are we ok to use Debian buster kernel (support notwithstanding)?

 

[t.lamprecht]

I wanted to test the btrfs raid1c3 feature, which was released on Kernel Version 5.5 but the profile is still missing?

Yeah, the btrfs-progs user space tool package was not yet updated, I see if a build of a newer version is not to much hassle...

 

[t.lamprecht]

What’s actually in the Proxmox-specific kernel? So far as I can tell from the repo it’s just a few patches like ACS.

Did you check out the whole repo, i.e., noticed that it uses a git submodule for the actual kernel?

In short, it's normally newer (buster is still on 4.19, the 5.10 from backports isn't really supported their either), OpenZFS support baked in, the kernel build-configuration is optimized for Proxmox VE (allowing running more Containers and better KVM support, among other things), fixing things like network bridge HW address (i.e., allowing PVE VMs actually have network).

if we don’t need those patches, are we ok to use Debian buster kernel (support notwithstanding)?

Proxmox VE is only tested with our kernel, and has some assumptions baked in for it, booting another kernel can break things or not, depending on how the other kernel is build and what your setup consists of.
So yes, support in the forum/enterprise is then senseless, and you may not get any help here - switching out the major core part of the distribution can just have too many effects to be able to meaningfully pin down most issues..

But yeah, we're not playing kernel police, it's software freedom after all, run whatever kernel you like...

 

[Ramalama]

@t.lamprecht could you probably update the git if you have time (newer kernels)

Or do we test 5.11.7 longer, till it gets an "stable" flag?

and i run into errors if i modify debian/rules (im adding options for removing modules/drivers to make the kernel a bit smaller), because there is an check in the Makefile. And im not clever enough to workaround it

Cheers

 

[t.lamprecht]

@t.lamprecht could you probably update the git if you have time (newer kernels)

what do you mean? git is up to date, all sources from our are pushed out (5.11 for 6.x is in the buster-pve-kernel-5.11 branch).

Or do we test 5.11.7 longer, till it gets an "stable" flag?

Proxmox VE 6.x will stay on 5.4 as stable default kernel, the 5.11 will only get introduced as default kernel with Proxmox VE 7.0 later this year. That said, I run the 5.11 on quite some HW too, and there's nothing obv. beta/faulty/... in it, so if you require it due to your HW just working better, or at all, with it I see not disadvantage using it.

But what would change if it gets a "stable flag"?

and i run into errors if i modify debian/rules (im adding options for removing modules/drivers to make the kernel a bit smaller), because there is an check in the Makefile. And im not clever enough to workaround it

That's possible better discussed in its own thread (or even better, mailing list) and the actual diff of your changes posted.
Note, if you want to tinker, that's cool and fine, but we can only provide some best effort help there for specific questions. Debian and Kernel developer and build documentation should get you a long way; albeit it can be quite the reading material.
In that case you'd just need to add the respective KConfig option with a -d (disable) -e (enable) -m (enable but as module), and so on flags to the PVE_CONFIG_OPTS variables.

 

[Ramalama]

what do you mean? git is up to date, all sources from our are pushed out (5.11 for 6.x is in the buster-pve-kernel-5.11 branch).

I mean 5.11.11 for example. Or to be more clear, to update the mirror_ubuntu-hirsute-kernel.git (Mirror checkout commit)
I don't know, how to do it locally, with git, maybe thats the only issue here xD
Else i would have replaced the "e012fdfc5752......." commit.

Proxmox VE 6.x will stay on 5.4 as stable default kernel, the 5.11 will only get introduced as default kernel with Proxmox VE 7.0 later this year. That said, I run the 5.11 on quite some HW too, and there's nothing obv. beta/faulty/... in it, so if you require it due to your HW just working better, or at all, with it I see not disadvantage using it.

Yes, thats clear, i didn't mean to switch default to 5.11, its perfectly fine right now as it is. I meant simply the patchlevel .7, if this 5.11 branch is going to be updated too, or it stays longer on 5.11.7.

That's possible better discussed in its own thread (or even better, mailing list) and the actual diff of your changes posted.
Note, if you want to tinker, that's cool and fine, but we can only provide some best effort help there for specific questions. Debian and Kernel developer and build documentation should get you a long way; albeit it can be quite the reading material.
In that case you'd just need to add the respective KConfig option with a -d (disable) -e (enable) -m (enable but as module), and so on flags to the PVE_CONFIG_OPTS variables.

-d CONFIG_XEN \
-d CONFIG_DRM_RADEON \
-d CONFIG_DRM_AMDGPU \
-d CONFIG_DRM_NOUVEAU \
-d CONFIG_DRM_I915 \
-d CONFIG_DRM_XEN \
-d CONFIG_DRM_GMA500 \
-d CONFIG_SND_FIREWIRE \
-d CONFIG_SND_XEN_FRONTEND \
-d CONFIG_HYPERV \
-d CONFIG_SURFACE_PLATFORMS \
-d CONFIG_NET_VENDOR_GOOGLE \
-d CONFIG_CHROME_PLATFORMS \
-d CONFIG_MICROCODE_INTEL \
-d CONFIG_NFC \
-d CONFIG_MAC80211 \
-d CONFIG_CFG80211 \
-d CONFIG_BT \
-d CONFIG_RTL8192U \
-d CONFIG_RTLLIB \
-d CONFIG_WIMAX \
-d CONFIG_ANDROID \

Basically im adding this to debian/rules, and probably much more in future. And yes, that needs probably another thread.
Just wanted to mention that it won't let me compile, because inside debian/rules on line 254, there is a "fwcheck", which fails if i remove modules xD
However, thats not important at the moment anyway.

Thanks for the reply!

 

[t.lamprecht]

I mean 5.11.11 for example. Or to be more clear, to update the mirror_ubuntu-hirsute-kernel.git (Mirror checkout commit)
I don't know, how to do it locally, with git, maybe thats the only issue here xD
Else i would have replaced the "e012fdfc5752......." commit.

Ubuntu has no newer tagged release, so there's nothing to update too. What do you expect from working/changing with the slightly newer stable version? I suggest reading a git submodule tutorial, or the documentation:
https://git-scm.com/book/en/v2/Git-Tools-Submodules if you actually want to work with the kernel repo. Basically you just need to cd into the respective submodule directory, add a kernel git remote (Ubuntus, or stable Linux tree) and rebase/cherry-pick whatever commits you like.

Yes, thats clear, i didn't mean to switch default to 5.11, its perfectly fine right now as it is. I meant simply the patchlevel .7, if this 5.11 branch is going to be updated too, or it stays longer on 5.11.7.

As soon as there's an actual release tagged we will update, sooner or later. Now, that this is only as opt-in and not default we do not invest to much time in it, as minor releases often have a "bad ROI", i.e., lots of work to package and test but not much fixed/improved (its always different, but often there are just patches for other architectures like POWER, SPARC or arm or the like)). We try to distribute our work time such that all features and subprojects get the attention needed, build just every minor kernel release (there are weeks with four in them) is just to much overhead.

Just wanted to mention that it won't let me compile, because inside debian/rules on line 254, there is a "fwcheck", which fails if i remove modules xD

You probably can just disable the fwcheck for self-builds, it is for our buildsystem to keep pve-firmware upto date if there's new firmware a module can use.
I'd not disable the ABI check, though, that is good for everybody to ensure modules have a useable ABI. If that fails you can normally just bump the kernel local release version for your build (or see what changed why).

 

[comfreak]

Yeah, the btrfs-progs user space tool package was not yet updated, I see if a build of a newer version is not to much hassle...

That would be great. Its the only reason why I wanted to test the new kernel as I am waiting for the raid1c3 profile

 

[Ramalama]

Ubuntu has no newer tagged release, so there's nothing to update too. What do you expect from working/changing with the slightly newer stable version? I suggest reading a git submodule tutorial, or the documentation:
https://git-scm.com/book/en/v2/Git-Tools-Submodules if you actually want to work with the kernel repo. Basically you just need to cd into the respective submodule directory, add a kernel git remote (Ubuntus, or stable Linux tree) and rebase/cherry-pick whatever commits you like.


As soon as there's an actual release tagged we will update, sooner or later. Now, that this is only as opt-in and not default we do not invest to much time in it, as minor releases often have a "bad ROI", i.e., lots of work to package and test but not much fixed/improved (its always different, but often there are just patches for other architectures like POWER, SPARC or arm or the like)). We try to distribute our work time such that all features and subprojects get the attention needed, build just every minor kernel release (there are weeks with four in them) is just to much overhead.


You probably can just disable the fwcheck for self-builds, it is for our buildsystem to keep pve-firmware upto date if there's new firmware a module can use.
I'd not disable the ABI check, though, that is good for everybody to ensure modules have a useable ABI. If that fails you can normally just bump the kernel local release version for your build (or see what changed why).

Thanks, i see it now, and yes hirsute isn't on 5.11.11 either.
I do not do it for Stability or any expectations, just want to understand it and probably i have a small up2date "tick" or Corona "nebenwirkungen" (Keine Ahnung wie das im Englischen heißt xD)
Let me say, i feel better if i compiled the kernel with O3 and znver3 and stripped it down as much as possible. I know that its probably waste of time and the actual perfomance difference is just my imagination and o3 probably even do things worse instead of better, but yeah xD

About the fwcheck, yes disabling brings me forward, but leads to a deb package that has only some kb. But doesn't matter, i will figure it out.

Thanks!

 

[t.lamprecht]

That would be great. Its the only reason why I wanted to test the new kernel as I am waiting for the raid1c3 profile

Actually, you could just install the newer version from Debian buster-backports, btrfs-progs 5.10.1-1~bpo10+1 is currently available from there.
btrfs-progs does feature checking for kernel support, so they do not need to explicitly match, or better said, nothing should break if they do not match 1:1 - https://btrfs.wiki.kernel.org/index...rfs-progs_at_the_same_version_as_my_kernel.3F