Windows server 2019 Critical structure corruption, nested virtualization

[Pietro395]

Decided to try the above scenario Linux Qemu-KVM>Windows Server>Virtualbox with different Qemu and Linux Kernel versions.
On the same AMD Ryzen 5 3600 CPU host.

Installed Debian 11 from scratch, the Qemu version is 5.2 Linux Kernel is 5.10.0-8-amd64
BSOD on Windows server guest running Virtualbox present.

Updated Qemu to version 6.1 from source code
BSOD also present

Updated Linux kernel to version 5.10.69 from source code with make olddefconfig
BSOD still present

Updated Linux Kernel to version 5.14.8 also with make olddefconfig
Windows server guest is already running with nested Virtualbox for more than 24 hours without BSOD.
So it looks like Linux Kernel 5.14.8 already has the fix for the above problem.

Will also try to update Linux Kernel to 5.14.8 on Proxmox7 host
Should I possibly get any problems with Proxmox functionality in this case?

Still looking for better and more simple solution.
If fix code applies only to kvm_amd module, maybe it is possible to install custom fixed kvm_amd module on Proxmox7.

@Dominic @aaron
Can we have some advice on this problem?

 

[stakoz]

Decided to try the above scenario Linux Qemu-KVM>Windows Server>Virtualbox with different Qemu and Linux Kernel versions.
On the same AMD Ryzen 5 3600 CPU host.

Installed Debian 11 from scratch, the Qemu version is 5.2 Linux Kernel is 5.10.0-8-amd64
BSOD on Windows server guest running Virtualbox present.

Updated Qemu to version 6.1 from source code
BSOD also present

Updated Linux kernel to version 5.10.69 from source code with make olddefconfig
BSOD still present

Updated Linux Kernel to version 5.14.8 also with make olddefconfig
Windows server guest is already running with nested Virtualbox for more than 24 hours without BSOD.
So it looks like Linux Kernel 5.14.8 already has the fix for the above problem.

Will also try to update Linux Kernel to 5.14.8 on Proxmox7 host
Should I possibly get any problems with Proxmox functionality in this case?

Still looking for better and more simple solution.
If fix code applies only to kvm_amd module, maybe it is possible to install custom fixed kvm_amd module on Proxmox7.

@apkoval Thanks a lot for the suggestion. Recently I was testing Windows Server 2022 with Hyper-V role (which now officially supports nested virtualization on AMD) on a Proxmox 7 with R9-5950X build (BIOS/AGESA up to date). While it worked just fine with legacy Gen1 VMs, it would always stuck at the boot and eventually crash the Gen2 guest (Hyper-V host was stable). Then I saw your message and decided to give PVE Edge[1] kernel 5.14 a try, and it worked fine[2] without any tweaks. I installed another instance of Windows 2022 (L2) inside Windows 2022 (L1) and it's running fine so far.

@Dominic It seems kernel 5.14 has some kvm-amd improvements that might be worth backporting.

[1] https://github.com/fabianishere/pve-edge-kernel
[2] You will need to apply a fix for AppArmor issue they document on their page

 

[Pietro395]

I also installed the 5.14 kernel on Proxmox 6 and I'm not having any problems

 

[stakoz]

A quick update. I have updated my Ryzen host to Proxmox 7.1 with kernel 5.13 and now it failed even to boot Windows Server 2022 VM with Hyper-V role enabled. Previously, Proxmox 7.0 would boot Hyper-V enabled VM just fine, but nested virtualization would work only for Gen 1 VMs. Currently staying on a combination of Proxmox 7.1 and kernel 5.14-edge, which works flawlessly in terms of nested virtualization on Ryzen CPUs.

 

[apkoval]

I have updated the Ryzen hosts to Proxmox 7.1 with 5.13.19-2-pve Kernel. I can confirm that in my use case the issue is fixed now. Proxmox7.1>Windows Server>Virtualbox nested virtualization works, no more BSODs on Windows VMs.