Compile custom kernel to fix AMD issue with nested ESXi VMs

T

tchuyev

Member
Dec 31, 2022
19
11
8
Hi,

Since kernel 5.16, it is no longer possible to run VMs on top of nested ESXi servers (7.0 or 8.0) on an AMD platform.
Apparently, this bug has been fixed but is still not applied in latest kernel e.g. pve-kernel-6.2.16-3-pve

https://git.kernel.org/pub/scm/linu.../?id=174a921b6975ef959dd82ee9e8844067a62e3ec1

Can someone guide me on how to build my own Linux custom kernel on PVE 8.0 (Debian 12)?
I've Googled and found way too many documents on the subject... and I really don't know where to start.

In a nutshell:
1. What packages/dependencies should I add to PVE to be able to build a custom kernel?
2. How to apply fix 174a921b6975ef959dd82ee9e8844067a62e3ec1 during compilation?

I could obviously pin 5.15 Proxmox Kernel Image at boot but the idea is to benefit from kernel 6.2+ features.

Any help would be greatly appreciated.
Thank you.
 
It must be a different fix despite kind of similar I'm afraid.
Nested VMs won't boot with kernel above 5.15 on AMD.
Easy to reproduce.

# cat /sys/module/kvm_amd/parameters/nested
1

# apt install bpftrace
# bpftrace -e 'kprobe:__nested_vmcb_check_controls { printf("tlb_ctl: %d\n", *((uint8 *)arg1+60) )}'

Results show issue is still present.

Any chance to get some help on how to build my own custom kernel with the fix?
Thank you so much
 
Last edited:
Hi,
the commit 174a921b6975ef959dd82ee9e8844067a62e3ec1 is included in all recent Proxmox VE (opt-in) kernels. Why do assume it is the fix for your issue?
Code: 
febner@pve7-dev ..on/pve-kernel/submodules/ubuntu-kernel % git tag --contains 174a921b6975ef959dd82ee9e8844067a62e3ec1
Proxmox-6.1.10-1
Proxmox-6.1.14-1
Proxmox-6.1.15-1
Proxmox-6.2.11-1
Proxmox-6.2.9-1
Ubuntu-5.19.0-24.25
Ubuntu-5.19.0-31.32
Ubuntu-6.1.0-1.1
Ubuntu-6.1.0-12.12
Ubuntu-6.1.0-13.13
Ubuntu-6.1.0-14.14
Ubuntu-6.2.0-16.16
Ubuntu-6.2.0-17.17
Ubuntu-6.2.0-19.19
v5.19
v6.1
 
Issue is finally fixed in Proxmox 8.1.x (kernel 6.5.x): VMs can now be created/ran on top of nested ESXi 8.x hypervisors.
Working using PVE No-Subscription repository.
That's great news :)
 
You must log in or register to reply here.
Top Bottom
Back