Nested virtualization + GPU Passthrough Causes VM Crashes/Bluescreens

NojuHD

Member
May 1, 2022
41
8
13
Hello Guys,

I have a Windows Server 2022 VM with GPU passthrough, so now i wanted to use WSL and for that i need Hyper-V.

Logical solution was to enable Nested Virtualization by editing the /etc/modprobe.d/kvm_amd.conf file and adding following parameters options kvm-amd nested=1 avic=1 npt=1 sev=0

My /etc/modprobe.d/kvm_amd.conf file was looking like this before the changes: options kvm-amd nested=0 avic=1 npt=1

But after restarting my Host and starting the VM i get Bluescreens on the start or sometimes a bit later after logging in.

After removing the GPU Passthrough it worked fine and i could install WSL2.

options kvm ignore_msrs=1 is also enabled :)

This is my VM Config:
agent: 1,fstrim_cloned_disks=1 args: -cpu host,hv_ipi,hv_relaxed,hv_reset,hv_runtime,hv_spinlocks=0x1fff,hv_stimer,hv_synic,hv_time,hv_vapic,hv-avic=on,hv_vpindex,+kvm_pv_eoi,+kvm_pv_unhalt,-hypervisor,+svm bios: ovmf boot: order=virtio0;ide2;net0 cores: 8 cpu: host efidisk0: ZFS-DATA-01:vm-7010-disk-0,efitype=4m,pre-enrolled-keys=1,size=1M #hostpci0: 0000:5d:00.0,pcie=1 machine: pc-q35-7.1 memory: 8192 meta: creation-qemu=7.1.0,ctime=1670887634 name: TS-01 net0: virtio=xxx,bridge=vmbr70,firewall=1 numa: 1 onboot: 1 ostype: win11 scsihw: virtio-scsi-single smbios1: uuid=xxx sockets: 1 startup: down=600 tpmstate0: ZFS-DATA-01:vm-7010-disk-1,size=4M,version=v2.0 virtio0: ZFS-DATA-01:vm-7010-disk-2,discard=on,iothread=1,size=32G vmgenid: xxx


When i remove the Graphics Driver the VM Boots, but as soon as i install the Driver and do a restart it wont work :(

1671165894495.png

Can someone help me on that?

Thanks in advance :)
 
Last edited:
I have a similar problem with vGPU and WSL. If I boot the VM with -cpu HOST ie for nested virtualization to work so that WSL ubuntu works inside the VM, it only works if I disable the GPU in the VM, otherwise when the Nvidia driver loads in the VM, it crashes the VM.
But if I disable nested virtualization, ie use -cpu kvm64 instead of -cpu HOST in proxmox settings for the VM. Then my GPU works fine in VM, but WSL/Ubuntu will not work...

https://forum.proxmox.com/threads/l...ws-guest-vm-crash-nvlddmkm-sys-failed.119425/
https://forum.proxmox.com/threads/kernel-5-11-nvidia-linux-vgpu-kvm.94266/page-2