[TUTORIAL] Building the PVE kernel on Proxmox VE 6.x

Thank you that moved things forward, I see I have a full copy of a version of ubuntu kernel in the folder which is not 5.3GB

However

mk-build-deps --install debian/control.in

Still fails with the same error

Code: 
dpkg-buildpackage: info: source version unknown
dpkg-buildpackage: error: version number does not start with digit
Error in the build process: exit status 255
dpkg: error: cannot access archive 'proxmox-kernel-@KVMAJMIN@-build-deps_1.0_all.deb': No such file or directory
mk-build-deps: dpkg --unpack failed

Is this a real bug with the sources ?

I searched the whole tree with find | grep \\.deb , there are no deb files

What is that file ?
 
you need to generate the real control file - @t.lamprecht just pushed a change that allows you to generate that more easily without parsing arcane Makefile syntax ;)
 
If you truly need to build a custom kernel you could use either:

Do you really need to compile the kernel (because you are applying a patch) or can use dkms to compile just the module you need from guthub?

This is how i built an ubuntu 6.5.2 kernel which i needed due to the diff patches i was testing, IMO if you are building, try using the pve kerenl source instead - but the process should be the same https://forum.proxmox.com/threads/t...link-state-change-aborting.133104/post-587604
 
fabian said:
just pushed a change that allows you to generate that more easily without parsing arcane Makefile syntax
Click to expand...
cool, i generated mine by pulling off a running config (/boot/config-6.2.16-10-pve) is that a bad idea?
 
@scyto
This is about the debian/control.in file

I finally found out that you need to run the command

make debian.prepared

From the /usr/src/pve-kernel

A command so obscure, it has never been mentionned ever on this forum, duckduckgo nor google !

1695161150174.pngNow I could finally successfully run

Code: 
root@srv4:/usr/src/pve-kernel# mk-build-deps -ir ./proxmox-kernel-6.2.16/debian/control

And then copy over the config file in the kernel sources, I don't know when I created this file, possibly when I ran make submodules

Code: 
root@srv4:/usr/src/pve-kernel# cp proxmox-kernel-6.2.16/ubuntu-kernel/.config proxmox-kernel-6.2.16/ubuntu-kernel/.config.bak
root@srv4:/usr/src/pve-kernel# cp config-6.2.16.org proxmox-kernel-6.2.16/ubuntu-kernel/.config
root@srv4:/usr/src/pve-kernel# cd  proxmox-kernel-6.2.16/ubuntu-kernel/
root@srv4:/usr/src/pve-kernel/proxmox-kernel-6.2.16/ubuntu-kernel# make menuconfig

And there it is !
 

Attachments

  • 1695161136260.png
    1695161136260.png
    20.6 KB · Views: 15
shodan said:
This is about the debian/control.in file
Click to expand...
these? https://www.debian.org/doc/debian-policy/ch-controlfields.html

I am intrigued as i never had to run `mk-build-deps -ir ./proxmox-kernel-6.2.16/debian/control` to compile a kernel from ubuntu or linux kernel nor did i copy the config from the kernel sources - i copied the running config and answered all the other questions using make oldedefconfig (avoiding menuconfig all together as i had no idea how to answer the questions)

I am trying to understand the root need of you taking a different path so i understand more about kernel building.
 
I was going from the suggestion of someone earlier in this thread

Presumably this replace this step

sudo apt-get install libncurses5-dev gcc make git exuberant-ctags bc libssl-dev build-essential wget bison flex libncurses-dev libelf-devmake rsync zstd debhelper

And installs only the packages which have to be installed, rather than a static list of packages ?

It might do other things, I am unsure.

make debian.prepared appears to have done a lot of stuff

My build eventually failed, due to a missing certificate

Code: 
CC      certs/system_keyring.o
make[2]: *** No rule to make target 'debian/canonical-certs.pem', needed by 'certs/x509_certificate_list'.  Stop.
make[1]: *** [scripts/Makefile.build:512: certs] Error 2
make: *** [Makefile:2026: .] Error 2

I haven't yet search why, maybe for secure boot ?

https://stackoverflow.com/questions...le-to-make-target-debian-certs-debian-uefi-ce

I had a look and the file is not in my /usr/src

Code: 
root@srv4:/usr/src/pve-kernel/proxmox-kernel-6.2.16/ubuntu-kernel# find /usr/src/ | grep -i \\.pem
/usr/src/pve-kernel/submodules/ubuntu-kernel/tools/testing/selftests/sgx/sign_key.pem
/usr/src/pve-kernel/submodules/ubuntu-kernel/debian/revoked-certs/canonical-uefi-2018-all.pem
/usr/src/pve-kernel/submodules/ubuntu-kernel/debian/revoked-certs/canonical-uefi-2021v3-all.pem
/usr/src/pve-kernel/submodules/ubuntu-kernel/debian/revoked-certs/canonical-uefi-2021v1-all.pem
/usr/src/pve-kernel/submodules/ubuntu-kernel/debian/revoked-certs/canonical-uefi-uc2019-all.pem
/usr/src/pve-kernel/submodules/ubuntu-kernel/debian/revoked-certs/canonical-uefi-2012-all.pem
/usr/src/pve-kernel/submodules/ubuntu-kernel/debian/revoked-certs/canonical-uefi-2019-all.pem
/usr/src/pve-kernel/submodules/ubuntu-kernel/debian/revoked-certs/canonical-uefi-2021v2-all.pem
/usr/src/pve-kernel/submodules/ubuntu-kernel/debian/revoked-certs/canonical-uefi-2017-all.pem
/usr/src/pve-kernel/submodules/ubuntu-kernel/debian/certs/canonical-livepatch-all.pem
/usr/src/pve-kernel/submodules/ubuntu-kernel/debian/certs/ubuntu-drivers-all.pem
/usr/src/pve-kernel/proxmox-kernel-6.2.16/ubuntu-kernel/tools/testing/selftests/sgx/sign_key.pem

Maybe if I link to /usr/src/pve-kernel/submodules/ubuntu-kernel/debian/certs/canonical-livepatch-all.pem that would be good enough, I'll have to see whether I can find the real file somewhere
 
Thanks, i will take a look and see if that file you are having issues with is my ubuntu source i used or not.

I might be tempted to use the pve source instead of the ubuntu source and see if what i did worked... esp now my patches I need are in 6.2.16-14-pve (aka i don't need to patch just try building)
 
Another thing to add before compiling

In the .config file, you have to remove the extra keys from canonical

change

CONFIG_SYSTEM_TRUSTED_KEYS="debian/canonical-certs.pem"
CONFIG_SYSTEM_REVOCATION_KEYS="debian/canonical-revoked-certs.pem"

to

CONFIG_SYSTEM_TRUSTED_KEYS=""
CONFIG_SYSTEM_REVOCATION_KEYS=""


also building the kernel much faster by allowing more core since the default strangely is not to use all cores, but to use only one !

make -j 100

Now I'm trying to figure out how to properly install my kernel and modules in a way that doesn't break proxmox-boot-tool !

I suspect that the old

Code: 
make install
make modules_install

Are going to break things !

Also, I noticed I have a /boot folder that is not my mounted EFI system partition, and it's full of stuff

I'm not sure if I should have the /boot mounted all the time, if it's ok to delete everything in /boot of the root partition and only the EFI ESP counts ?!

Once I've got all that, I'll make a final post that describe building and instaling the kernel from a fresh install
 
The .config file I have used is

/usr/src/pve-kernel/config-6.2.16.org

I'm not sure when I picked up that file, it was one of the commands I ran.

Here is a copy after I have removed the two .pem files

https://pastebin.com/inT7cpQc

The PVE kernel readme https://git.proxmox.com/?p=pve-kern...5dc8aa399ba86a0e846cd1837a7c19d3da48e;hb=HEAD

Says to check for the following settings

Code: 
cat proxmox-kernel-6.2.16/ubuntu-kernel/.config | grep INTEL_MEI_WDT
cat proxmox-kernel-6.2.16/ubuntu-kernel/.config | grep CONFIG_SND_PCM_OSS
cat proxmox-kernel-6.2.16/ubuntu-kernel/.config | grep CONFIG_TRANSPARENT_HUGEPAGE
cat proxmox-kernel-6.2.16/ubuntu-kernel/.config | grep CONFIG_CEPH_FS=
cat proxmox-kernel-6.2.16/ubuntu-kernel/.config | grep CONFIG_BLK_DEV_SD
cat proxmox-kernel-6.2.16/ubuntu-kernel/.config | grep CONFIG_BLK_DEV_SR
cat proxmox-kernel-6.2.16/ubuntu-kernel/.config | grep CONFIG_BLK_DEV_DM
cat proxmox-kernel-6.2.16/ubuntu-kernel/.config | grep CONFIG_BLK_DEV_NBD
cat proxmox-kernel-6.2.16/ubuntu-kernel/.config | grep CONFIG_BLK_DEV_RBD
cat proxmox-kernel-6.2.16/ubuntu-kernel/.config | grep CONFIG_JFS_FS
cat proxmox-kernel-6.2.16/ubuntu-kernel/.config | grep CONFIG_HFS_FS
cat proxmox-kernel-6.2.16/ubuntu-kernel/.config | grep CONFIG_HFSPLUS_FS
cat proxmox-kernel-6.2.16/ubuntu-kernel/.config | grep CONFIG_BCACHE=
cat proxmox-kernel-6.2.16/ubuntu-kernel/.config | grep CONFIG_BRIDGE=
cat proxmox-kernel-6.2.16/ubuntu-kernel/.config | grep CONFIG_DEFAULT_SECURITY_APPARMOR
cat proxmox-kernel-6.2.16/ubuntu-kernel/.config | grep CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE
cat proxmox-kernel-6.2.16/ubuntu-kernel/.config | grep -i CONFIG_MQ_IOSCHED
cat proxmox-kernel-6.2.16/ubuntu-kernel/.config | grep CONFIG_INPUT_EVBUG
cat proxmox-kernel-6.2.16/ubuntu-kernel/.config | grep CONFIG_MODVERSIONS
cat proxmox-kernel-6.2.16/ubuntu-kernel/.config | grep UNWINDER
cat proxmox-kernel-6.2.16/ubuntu-kernel/.config | grep FRAME_POINTER
cat proxmox-kernel-6.2.16/ubuntu-kernel/.config | grep CONFIG_PAGE_TABLE_ISOLATION

my results were

Code: 
CONFIG_INTEL_MEI_WDT=m
# CONFIG_SND_PCM_OSS is not set
CONFIG_TRANSPARENT_HUGEPAGE=y
# CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS is not set
CONFIG_TRANSPARENT_HUGEPAGE_MADVISE=y
CONFIG_CEPH_FS=m
CONFIG_BLK_DEV_SD=y
CONFIG_BLK_DEV_SR=y
CONFIG_BLK_DEV_DM_BUILTIN=y
CONFIG_BLK_DEV_DM=y
CONFIG_BLK_DEV_NBD=m
CONFIG_BLK_DEV_RBD=m
CONFIG_JFS_FS=m
CONFIG_HFS_FS=m
CONFIG_HFSPLUS_FS=m
CONFIG_BCACHE=m
CONFIG_BRIDGE=m
CONFIG_DEFAULT_SECURITY_APPARMOR=y
# CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE is not set
CONFIG_MQ_IOSCHED_DEADLINE=y
CONFIG_MQ_IOSCHED_KYBER=m
CONFIG_INPUT_EVBUG=m
CONFIG_MODVERSIONS=y
# CONFIG_UNWINDER_ORC is not set
CONFIG_UNWINDER_FRAME_POINTER=y
CONFIG_SCHED_OMIT_FRAME_POINTER=y
CONFIG_FRAME_POINTER=y
CONFIG_UNWINDER_FRAME_POINTER=y
CONFIG_PAGE_TABLE_ISOLATION=y

The kernel readme says

Code: 
NOTE: For the exact and current list see debian/rules (PVE_CONFIG_OPTS)

- enable INTEL_MEI_WDT=m (to allow disabling via patch)

- disable CONFIG_SND_PCM_OSS (enabled by default in Ubuntu, not needed)

- switch CONFIG_TRANSPARENT_HUGEPAGE to MADVISE from ALWAYS

- enable CONFIG_CEPH_FS=m (request from user)

- enable common CONFIG_BLK_DEV_XXX to avoid hardware detection
problems (udev, update-initramfs have serious problems without that)

CONFIG_BLK_DEV_SD=y
CONFIG_BLK_DEV_SR=y
CONFIG_BLK_DEV_DM=y

- compile NBD and RBD modules
CONFIG_BLK_DEV_NBD=m
CONFIG_BLK_DEV_RBD=m

- enable IBM JFS file system as module
requested by users (bug #64)

- enable apple HFS and HFSPLUS as module
requested by users

- enable CONFIG_BCACHE=m (requested by user)

- enable CONFIG_BRIDGE=y
to avoid warnings on boot, e.g. that net.bridge.bridge-nf-call-iptables is an unknown key

- enable CONFIG_DEFAULT_SECURITY_APPARMOR
We need this for lxc

- set CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE=y
because if not set, it can give some dynamic memory or cpu frequencies
change, and vms can crash (mainly windows guest).
see http://forum.proxmox.com/threads/18238-Windows-7-x64-VMs-crashing-randomly-during-process-termination?p=93273#post93273

- use 'deadline' as default scheduler
This is the suggested setting for KVM. We also measure bad fsync performance with ext4 and cfq.

- disable CONFIG_INPUT_EVBUG
Module evbug is not blacklisted on debian, so we simply disable it to avoid
key-event logs (which is a big security problem)

- enable CONFIG_MODVERSIONS (needed for ABI tracking)

- switch default UNWINDER to FRAME_POINTER
the recently introduced ORC_UNWINDER is not 100% stable yet, especially in combination with ZFS
- enable CONFIG_PAGE_TABLE_ISOLATION (Meltdown mitigation)


So the following are wrong in the file config-6.2.16.org

Code: 
CONFIG_INPUT_EVBUG=m
# CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE is not set
CONFIG_BRIDGE=m



Hmm I am curious it says "For the exact and current list see debian/rules (PVE_CONFIG_OPTS)"

There is this file https://git.proxmox.com/?p=pve-kern...31d4e3749014d5e27d8104c7da4f3b75a636c;hb=HEAD

In that file it says

Code: 
-d CONFIG_INPUT_EVBUG \
-e CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE
-e CONFIG_BRIDGE

I'm going to assume that -d means disable and -e means enable

Code: 
CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE=y
CONFIG_INPUT_EVBUG=n
CONFIG_BRIDGE=y
 
Last edited:
shodan said:
A command so obscure, it has never been mentionned ever on this forum, duckduckgo nor google !
Click to expand...
It's simply calling a makefile target, and they can be named whatever, so I'd not call make obscure..

scyto said:
cool, i generated mine by pulling off a running config (/boot/config-6.2.16-10-pve) is that a bad idea?
Click to expand...
No, it's a good idea. It's also documented as such in the README.
shodan said:
Says to check for the following settings
Click to expand...
The canonical source of KCOnfig switches we override on top of the ubuntu defaults is in the debian/rules file, which is basically the main makefile that the actual debian packaging process (dpkg-buildpackage) is using.
shodan said:
I'm going to assume that -d means disable and -e means enable
Click to expand...
Yes, at this level most is working directly on the actual kernel source, so it's just calling scripts/config (i.e., the main Linux KConfig management tool), which outputs a help:

Code: 
Manipulate options in a .config file from the command line.
Usage:
config options command ...
commands:
        --enable|-e option   Enable option
        --disable|-d option  Disable option
        --module|-m option   Turn option into a module
        --set-str option string
                             Set option to "string"
        --set-val option value
                             Set option to value
        --undefine|-u option Undefine option
        --state|-s option    Print state of option (n,y,m,undef)

        --enable-after|-E beforeopt option
                             Enable option directly after other option
        --disable-after|-D beforeopt option
                             Disable option directly after other option
        --module-after|-M beforeopt option
                             Turn option into module directly after other option

        commands can be repeated multiple times

options:
        --file config-file   .config file to change (default .config)
        --keep-case|-k       Keep next symbols' case (dont' upper-case it)

config doesn't check the validity of the .config file. This is done at next
make time.

By default, config will upper-case the given symbol. Use --keep-case to keep
the case of all following symbols unchanged.

config uses 'CONFIG_' as the default symbol prefix. Set the environment
variable CONFIG_ to the prefix to use. Eg.: CONFIG_="FOO_" config ...
 
  • Like
Reactions: scyto
Here is my trial-and-error way of compiling the pve kernel on a fresh Proxmox VE 8.0.4 ( 6.2.16-3-pve )

Code: 
apt update
apt install asciidoc-base automake bc bison cpio debhelper dh-python dwarves file flex gcc git kmod libdw-dev libelf-dev libiberty-dev libnuma-dev libpve-common-perl libslang2-dev libssl-dev libtool lintian lz4 python3-minimal rsync sphinx-common xmlto zlib1g-dev zstd # build tools
apt install abigail-tools libaio-dev libblkid-dev libpam0g-dev libudev-dev lsb-release python3-cffi uuid-dev python3-all-dev # zfs tools
apt remove libcurl4-openssl-dev # fix package conflict
apt install libcurl4-gnutls-dev # fix package conflict

git clone -b bookworm-6.2 git://git.proxmox.com/git/pve-kernel.git
cd pve-kernel
nano patches/kernel/0020-custom.patch # insert own kernel patch

git submodule foreach git fetch --tags
git submodule update --init
cd submodules/zfsonlinux
make
cd ../..
make submodule
make

dpkg -i *.deb
reboot
apt-mark hold linux-image-*
apt-mark hold linux-headers-*
apt-mark hold proxmox-headers-*
apt-mark hold proxmox-kernel-*
 
Last edited:
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back