Proxmox kernel vs Debian 12 Stock - ramifications of staying with Stock

[VA1DER]

I'm looking at the instructions for installing Proxmox on top of an existing Debian 12 system. In section 3.2 it mentions installing a custom Proxmox kernel, with this explanation:

First you need to install and boot the Proxmox VE kernel, as some packages depend on specific kernel compile flags to be set or feature extensions (e.g., for apparmor) to be available.

The example given is puzzling in that the stock Debian kernel has had apparmor enabled for eleven years.

What compile flags are spoken of above? What feature extensions? Is the Proxmox kernel actually hard-required for Proxmox to function? Will only certain things not function without it? If so what?

If someone knows where this is documented that I'd really appreciate it. I grepped around the forum and wiki. I found the Proxmox kernel page, but it is silent on what changes are made to the kernel from stock and why.

Thanks!

 

[gfngfn256]

I can't directly answer your questions or concerns, but AFAIK in principle Proxmox uses a sort of "hybrid" approach, where the OS/packages are Debian & the Kernel is Ubuntu. The reasoning behind the kernel choice is because the Debian one gets outdated pretty fast & does not receive timely updates or backports vs the constantly updated Ubuntu one.

 

[UdoB]

The reasoning behind the kernel choice is because the Debian one gets outdated pretty fast

And there is no ZFS baked in, is it? And ZFS is one of the very best elements of PVE

 

[ness1602]

Not necessarily, if you use ext4 local disks, or lets say hardware raid, you dont need it.

 

[UdoB]

Not necessarily, if you use ext4 local disks, or lets say hardware raid, you dont need it.

Sure. But I am a ZFS fan..., and I need it.

Actually I would try hard not to run any PVE on non-ZFS. But yes, that's my choice and yours (and OP's) mileage may vary

 

[love4taylor]

And there is no ZFS baked in, is it? And ZFS is one of the very best elements of PVE

If I'm not mistaken, the reason why most distributions do not have ZFS built-in is due to issues with the open-source license.

 

[louie1961]

Proxmox without ZFS is sort of like kissing your sister. It's Just not the same.

 

[ness1602]

Yeah, SDDL is compatible with kernel, but kernel isnt compatible with it(GPL2).

 

[louie1961]

If I'm not mistaken, the reason why most distributions do not have ZFS built-in is due to issues with the open-source license.

"There is no way I can merge any of the ZFS efforts until I get an official letter from Oracle that is signed by their main legal counsel or preferably by Larry Ellison himself that says that yes, it's OK to do so and treat the end result as GPL'd." -Linus Torvalds circa 2020

https://www.theregister.com/2020/01/13/zfs_linux/

 

[leesteken]

If you keep the Debian kernel installed, you might run into issues with updates: https://forum.proxmox.com/threads/h...error-on-pve-manager-8-2-7.159045/post-729332

 

[VA1DER]

Thank-you for the replies. ZFS is beyond my needs. My PVE install is on a smallish, somewhat older data-center server and is for personal/family/donation use. I don't see a strong enough use case for ZFS to warrant replacing a distro's kernel with a vendor-bake.

you might run into issues with updates: https://forum.proxmox.com/threads/h...error-on-pve-manager-8-2-7.159045/post-729332

Thank-you for that link. I suspect that was to dissuade me, but it was actually quite comforting. It appears that thread's OP had a Debian kernel that worked quite well and only ran into issues with dependencies. I'm quite comfortable taking the dependency issue in hand.

My install with a Debian kernel appears to function properly. I'll certainly post if (or when) I run into issues.

 

[spirit]

Personally, you should use the proxmox provided kernel, because I have already seen kvm kernel bug/regression in past in debian kernel, and it take years to get it fixed.

Proxmox use ubuntu kernel and but also add patches on top of it, for security bugs, kvm bugs, lxc bugs.

 

[leesteken]

I suspect that was to dissuade me, but it was actually quite comforting.

I don't care either way, I just wanted to note that it's not theoretical that you might get package conflicts when the Debian kernel package is installed.

It appears that thread's OP had a Debian kernel that worked quite well and only ran into issues with dependencies.

They had the Debian kernel packages installed but they were running the Proxmox kernel. I don't think you can conclude anything about the working of that kernel since it was not used.

 

[louie1961]

ZFS is beyond my needs. My PVE install is on a smallish, somewhat older data-center server and is for personal/family/donation use. I don't see a strong enough use case for ZFS to warrant replacing a distro's kernel with a vendor-bake.

No offense and do as you please, but I think you have it completely backwards. The Proxmox kernel is half the reason for using Proxmox. Otherwise you could just run KVM/QEMU in Debian without Proxmox. Proxmox is well documented and the kernel every bit as stable as Debian's

 

[gfngfn256]

The above posted comments have basically covered the pitfalls of the OP's intended attitude of using a Debian kernel instead of the Proxmox provided one.

I'll only add, that while Proxmox is opensource by nature & therefore "an each to his own" method applies, so you can basically feel free to do what you want with this software (with in certain boundaries), however you are not to be considered running a PVE system.

Therefore in the future, when you have issues, which I'm pretty sure you will, any posts on this forum, (in my humble opinion) should contain the fact that you have carved up the PVE system by using your own/Debian kernel. Other readers must be made aware of this fact as your situation will be different to what they can expect on their systems.

Don't take this the wrong way - but if I could "dis-like" a post, the OP would have earned one from myself.

 

[VA1DER]

Therefore in the future, when you have issues, which I'm pretty sure you will, any posts on this forum, (in my humble opinion) should contain the fact that you have carved up the PVE system by using your own/Debian kernel.

I'm sure I'll run into questions. I'll be sure to note that in any other posts.

Understand, there are situations where one can't just run any kernel. The stock Debian kernel is one that is one that is vetted for certain installations, and vendor kenels like Proxmox's custom one isn't.

On a technical note, I solved the dependency issues so everything else except the custom kernel is updated nicely. I use a combination of pinning and a dummy package:

/etc/apt/preferences.d/debian-over-proxmox

Package: *
Pin: release o=Proxmox
Pin-Priority: 500

Package: *
Pin: release o=Debian
Pin-Priority: 600

Package: proxmox-default-kernel-dummy
Provides: proxmox-default-kernel
Priority: 1001

And a dummy package to take the kernel's place:

mkdir -p proxmox-default-kernel-dummy/DEBIAN
cat > proxmox-default-kernel-dummy/DEBIAN/control << EOF
Package: proxmox-default-kernel-dummy
Version: 1.0
Architecture: all
Provides: proxmox-default-kernel
Maintainer: Your Name <yourname@example.com>
Description: Dummy package to satisfy proxmox-default-kernel dependency
EOF
dpkg-deb --build proxmox-default-kernel-dummy

This seems to handle dependency issues quite nicely and Proxmox packages have so far been updating as expected.

Otherwise you could just run KVM/QEMU in Debian without Proxmox.

Proxmox still offers a reasonable UI for it, which is what I like. And it seems to be coming along nicely with btrfs support. It's not in the front-end yet, but there is back-end support for btrfs directories that will take advantage of snapshotting.

I can also note that my VMs and the front end all seem to run very well so far on the stock kernel. In case anyone else has a need or desire to do this. I think others in this thread have sufficiently pointed out why they think no one should.

 

[zodiac]

Understand, there are situations where one can't just run any kernel. The stock Debian kernel is one that is one that is vetted for certain installations, and vendor kenels like Proxmox's custom one isn't.

For all practical purposes, the Proxmox kernel is the same as the Ubuntu kernel. I have never encountered a situation that works with the Debian kernel, but not the Ubuntu kernel. I have seen a few commercial products, that will only work with ancient RedHat kernels. But those products wouldn't work with Debian either.

Also, it is strongly discouraged to modify the host system of a Proxmox VE installation. That is just asking for trouble. It will result in constant subtle breakage, and it makes disaster recovery a lot harder, too. How are you documenting all the non-standard customizations that you made, and that you need to restore when building a new node. For PVE, you want to run as much of an unmodified and minimal Debian system as you possibly can; and that's what the PVE installation media give you. If you need customizations or specific kernel versions, then you should do so in a VM.

I think we need to hear a lot more details on what it is you are actually trying to do. I have a feeling that we might be looking an an XY problem. Yes, it is technically possible to install the PVE packages on a Debian system installed from Debian media and with a Debian kernel. And if you take ownership of this new custom distribution, do regular QA testing, and maintain your own set of patches, then this can work. But it does require good familiarity with both Debian and PVE. This amount of recurring engineering effort is really only justified, if you have very specific needs that can't be met any other way. You very well might be. But then again, you might be looking at a solution without knowing what your problem is.

One particular pitfall is network configuration. PVE requires the ifupdown2 package that the Proxmox developers maintain. But with a normal Debian system, you can accidentally find yourself installing NetworkManager or systemd-networkd, and those will not work correctly with PVE at all. If I recall correctly, GNOME insists on using NetworkManager, unless you go out of your way to disable that dependency. This is just one of many examples, why it is risky to install PVE on anything other than a minimal base system.

 

[VA1DER]

For all practical purposes, the Proxmox kernel is the same as the Ubuntu kernel.

The issue is that in some situations, what matters is not the "for all practical purpose" but what has been actually been vetted and approved.

I think we need to hear a lot more details on what it is you are actually trying to do.

What I was trying to do was install PVE without it requiring the proxmox vendor-baked kernel.

The initial installation method I used was to simply download the "proxmox-ve" dependency package and remove the kernel dependency from its control file. I repacked it into a custom proxmox-ve package and compared what packages and changes apt wanted to make with the normal versus with my kernel-dependency-removed proxmox-ve package. I found the list was the same except, as desired, for the kernel and firmware packages.

That method worked very well for the initial installation, however it required forever maintaining a custom proxmox-ve dependency package which I didn't want to forever do. So I switched to the method I noted above with the dummy package to avoid this for updates and it seems to work to maintain the system with all changes except for the kernel.

One particular pitfall ... PVE requires the ifupdown2

With ifupdown2, as with every other PVE package that isn't a direct dependency of PVE's custom kernel (which as far as I can tell is only the kernel and firmware), the install method I indicated above pulled in the PVE ifupdown2 package (which caused the removal of the stock ifupdown) as per normal. My installation is, as far as I can tell, other than the kernel, identical to every other installation who uses the official Install on Debian method.

That said, this thread was intended to seek the answers to the questions "is this custom kernel actually strictly needed and if so why?" and "what happens if I don't use this custom kernel?", and not a "I'm going to prove I can automatically maintain this with no issues". This is an experiment I am conducting and I've given my results so far. Everyone is, of course, free to take these results for whatever value they feel they are worth or none at all.

 

[Boson]

I have never encountered a situation that works with the Debian kernel, but not the Ubuntu kernel.

Lucky you, because PVE kernel breaks my Thunderbolt 4 which works with Debian kernel without any issue, and that's how I found this thread.

 

[VA1DER]

Update: I've upgraded over four Proxmox minor revisions and two Debian point releases now, and the above method has worked flawlessly in keeping my systems up-to-date with every other Proxmox package and dependency.

For the purpose alone of retaining the Debian kernel, the dummy package should suffice and the repository pinning shouldn't be necessary. I keep the pinning in place in case a different package shows up in the Proxmox repository that wants to override a Debian package.