Nested virtualization + GPU Passthrough Causes VM Crashes/Bluescreens

May 1, 2022
25
2
3
Hello Guys,

I have a Windows Server 2022 VM with GPU passthrough, so now i wanted to use WSL and for that i need Hyper-V.

Logical solution was to enable Nested Virtualization by editing the /etc/modprobe.d/kvm_amd.conf file and adding following parameters options kvm-amd nested=1 avic=1 npt=1 sev=0

My /etc/modprobe.d/kvm_amd.conf file was looking like this before the changes: options kvm-amd nested=0 avic=1 npt=1

But after restarting my Host and starting the VM i get Bluescreens on the start or sometimes a bit later after logging in.

After removing the GPU Passthrough it worked fine and i could install WSL2.

options kvm ignore_msrs=1 is also enabled :)

This is my VM Config:
agent: 1,fstrim_cloned_disks=1 args: -cpu host,hv_ipi,hv_relaxed,hv_reset,hv_runtime,hv_spinlocks=0x1fff,hv_stimer,hv_synic,hv_time,hv_vapic,hv-avic=on,hv_vpindex,+kvm_pv_eoi,+kvm_pv_unhalt,-hypervisor,+svm bios: ovmf boot: order=virtio0;ide2;net0 cores: 8 cpu: host efidisk0: ZFS-DATA-01:vm-7010-disk-0,efitype=4m,pre-enrolled-keys=1,size=1M #hostpci0: 0000:5d:00.0,pcie=1 machine: pc-q35-7.1 memory: 8192 meta: creation-qemu=7.1.0,ctime=1670887634 name: TS-01 net0: virtio=xxx,bridge=vmbr70,firewall=1 numa: 1 onboot: 1 ostype: win11 scsihw: virtio-scsi-single smbios1: uuid=xxx sockets: 1 startup: down=600 tpmstate0: ZFS-DATA-01:vm-7010-disk-1,size=4M,version=v2.0 virtio0: ZFS-DATA-01:vm-7010-disk-2,discard=on,iothread=1,size=32G vmgenid: xxx


When i remove the Graphics Driver the VM Boots, but as soon as i install the Driver and do a restart it wont work :(

1671165894495.png

Can someone help me on that?

Thanks in advance :)
 
Last edited:
I have a similar problem with vGPU and WSL. If I boot the VM with -cpu HOST ie for nested virtualization to work so that WSL ubuntu works inside the VM, it only works if I disable the GPU in the VM, otherwise when the Nvidia driver loads in the VM, it crashes the VM.
But if I disable nested virtualization, ie use -cpu kvm64 instead of -cpu HOST in proxmox settings for the VM. Then my GPU works fine in VM, but WSL/Ubuntu will not work...

https://forum.proxmox.com/threads/l...ws-guest-vm-crash-nvlddmkm-sys-failed.119425/
https://forum.proxmox.com/threads/kernel-5-11-nvidia-linux-vgpu-kvm.94266/page-2
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!