Compile custom kernel to fix AMD issue with nested ESXi VMs

tchuyev

New Member
Dec 31, 2022
18
8
3
Hi,

Since kernel 5.16, it is no longer possible to run VMs on top of nested ESXi servers (7.0 or 8.0) on an AMD platform.
Apparently, this bug has been fixed but is still not applied in latest kernel e.g. pve-kernel-6.2.16-3-pve

https://git.kernel.org/pub/scm/linu.../?id=174a921b6975ef959dd82ee9e8844067a62e3ec1

Can someone guide me on how to build my own Linux custom kernel on PVE 8.0 (Debian 12)?
I've Googled and found way too many documents on the subject... and I really don't know where to start.

In a nutshell:
1. What packages/dependencies should I add to PVE to be able to build a custom kernel?
2. How to apply fix 174a921b6975ef959dd82ee9e8844067a62e3ec1 during compilation?

I could obviously pin 5.15 Proxmox Kernel Image at boot but the idea is to benefit from kernel 6.2+ features.

Any help would be greatly appreciated.
Thank you.
 
It must be a different fix despite kind of similar I'm afraid.
Nested VMs won't boot with kernel above 5.15 on AMD.
Easy to reproduce.

# cat /sys/module/kvm_amd/parameters/nested
1

# apt install bpftrace
# bpftrace -e 'kprobe:__nested_vmcb_check_controls { printf("tlb_ctl: %d\n", *((uint8 *)arg1+60) )}'

Results show issue is still present.

Any chance to get some help on how to build my own custom kernel with the fix?
Thank you so much
 
Last edited:
Hi,
the commit 174a921b6975ef959dd82ee9e8844067a62e3ec1 is included in all recent Proxmox VE (opt-in) kernels. Why do assume it is the fix for your issue?
Code:
febner@pve7-dev ..on/pve-kernel/submodules/ubuntu-kernel % git tag --contains 174a921b6975ef959dd82ee9e8844067a62e3ec1
Proxmox-6.1.10-1
Proxmox-6.1.14-1
Proxmox-6.1.15-1
Proxmox-6.2.11-1
Proxmox-6.2.9-1
Ubuntu-5.19.0-24.25
Ubuntu-5.19.0-31.32
Ubuntu-6.1.0-1.1
Ubuntu-6.1.0-12.12
Ubuntu-6.1.0-13.13
Ubuntu-6.1.0-14.14
Ubuntu-6.2.0-16.16
Ubuntu-6.2.0-17.17
Ubuntu-6.2.0-19.19
v5.19
v6.1
 
Issue is finally fixed in Proxmox 8.1.x (kernel 6.5.x): VMs can now be created/ran on top of nested ESXi 8.x hypervisors.
Working using PVE No-Subscription repository.
That's great news :)
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!