Can't run WSL2 on 8.1 (VIDEO_DXGKRNL_FATAL_ERROR)

[Lukium]

Hi everyone,

Whenever I run the CPU as host (needed for WSL2 afaik), I end up with the VM crashing every 11 minutes or so, or whenever I scan for hardware changes in the device manager. The BSOD is:

VIDEO_DXGKRNL_FATAL_ERROR​

I have tried googling around to no avail.
My setup:

CPU: Ryzen 9 5900x
MOBO: Asus Crosshair VIII Dark Hero
GPU: RTX 3090 (Passthrough enabled)

Settings (including everything I've changed to runt the VM):

qm config 101:

Spoiler

agent: 1
balloon: 0
bios: ovmf
boot: order=scsi0;net0
cores: 8
cpu: host
efidisk0: local-lvm:vm-101-disk-0,efitype=4m,pre-enrolled-keys=1,size=4M
hostpci0: mapping=RTX_3090_1,pcie=1
machine: pc-q35-8.1
memory: 16384
meta: creation-qemu=8.1.2,ctime=1700914965
name: pve-win-1
net0: virtio=BC:24:11:E5:16:F9,bridge=vmbr0,firewall=1
numa: 0
ostype: win11
scsi0: local-lvm:vm-101-disk-1,discard=on,iothread=1,size=128G,ssd=1
scsihw: virtio-scsi-single
smbios1: uuid=
sockets: 1
tpmstate0: local-lvm:vm-101-disk-2,size=4M,version=v2.0
vmgenid:

pveversion -v:

Spoiler

proxmox-ve: 8.1.0 (running kernel: 6.5.11-4-pve)
pve-manager: 8.1.3 (running version: 8.1.3/b46aac3b42da5d15)
proxmox-kernel-helper: 8.0.9
proxmox-kernel-6.5.11-4-pve-signed: 6.5.11-4
proxmox-kernel-6.5: 6.5.11-4
ceph-fuse: 17.2.7-pve1
corosync: 3.1.7-pve3
criu: 3.17.1-2
glusterfs-client: 10.3-5
ifupdown2: 3.2.0-1+pmx7
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-4
libknet1: 1.28-pve1
libproxmox-acme-perl: 1.5.0
libproxmox-backup-qemu0: 1.4.0
libproxmox-rs-perl: 0.3.1
libpve-access-control: 8.0.7
libpve-apiclient-perl: 3.3.1
libpve-common-perl: 8.1.0
libpve-guest-common-perl: 5.0.6
libpve-http-server-perl: 5.0.5
libpve-network-perl: 0.9.4
libpve-rs-perl: 0.8.7
libpve-storage-perl: 8.0.5
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 5.0.2-4
lxcfs: 5.0.3-pve3
novnc-pve: 1.4.0-3
proxmox-backup-client: 3.0.4-1
proxmox-backup-file-restore: 3.0.4-1
proxmox-kernel-helper: 8.0.9
proxmox-mail-forward: 0.2.2
proxmox-mini-journalreader: 1.4.0
proxmox-offline-mirror-helper: 0.6.2
proxmox-widget-toolkit: 4.1.3
pve-cluster: 8.0.5
pve-container: 5.0.8
pve-docs: 8.1.3
pve-edk2-firmware: 4.2023.08-1
pve-firewall: 5.0.3
pve-firmware: 3.9-1
pve-ha-manager: 4.0.3
pve-i18n: 3.1.2
pve-qemu-kvm: 8.1.2-4
pve-xtermjs: 5.3.0-2
qemu-server: 8.0.10
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.2.0-pve3

GRUB_CMDLINE_LINUX_DEFAULT:

Spoiler

quiet amd_iommu=on iommu=pt vfio-pci.ids=10de:2204,10de:147d disable_vga=1

/etc/modules:

Spoiler

# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.
# Parameters can be specified after the module name.

vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

/etc/modprobe.d/kvm.conf:

Spoiler

options kvm ignore_msrs=1
options kvm report_ignored_msrs=0

/etc/modprobe.d/blacklist.conf:

Spoiler

blacklist nouveau
blacklist nvidiafb

/etc/modprobe.d/vfio.conf

Spoiler

options vfio-pci ids=10de:2204,10de:147d disable_vga=1

I have tried turning off Secure Boot on the VM as well. I know that nested virtualization is working because:

cat /sys/module/kvm_amd/parameters/nested
1

and in the Windows Guest VM task manager shows Virtualization Enabled and I'm able to install ubuntu with wsl --set-default-version 2

And just in case it's a question, no, running a Linux Guest VM is not an option in this use case, because there are things that need to be run in the VM that are windows only.

Switching from HOST to any other CPU stops the BSODs from happening, but then WSL2 can't be used.

 

[Lukium]

I think I might have actually solved my problem.

It seems like the issue is that when I installed the VM, the CPU type was not set to HOST, and that broke the windows installation as far as setting it to host later.

EDIT: Actually this is not the issue. Debugging Further

 

[Lukium]

I have now been able to further narrow down the issue to a windows application that needs to use WSL2.

I started thinking that the issue had something to do with windows not being able to use hypervisor enlightenments, so I went ahead and set hv_passthrough=on for the CPU, and while it didn't solve the problem, it seems to have helped me further narrow down the issue to it likely being something to do with the hypervisor, because with hv_passthrough=on, the VM panics instead of getting a BSOD.

Does anyone have any suggestions on troubleshooting the VM panic?

Thanks in advance!

 

[eyeless77]

Hi. I had issues with nested virtualization on AMD Ryzen 5 3600x, somewhere I found a solution to add the following line to VM config (/etc/pve/qemu-server/vmid.conf)

args: -cpu 'host,+svm'

Give it a try.

 

[Lukium]

Hi. I had issues with nested virtualization on AMD Ryzen 5 3600x, somewhere I found a solution to add the following line to VM config (/etc/pve/qemu-server/vmid.conf)

args: -cpu 'host,+svm'

Give it a try.

Just gave it a go:

qm config 105
agent: 1
args: -cpu 'host,+svm'
balloon: 0
bios: ovmf
boot: order=ide2;scsi0;net0
cores: 8
cpu: host
efidisk0: local-lvm:vm-105-disk-0,efitype=4m,pre-enrolled-keys=1,size=4M
hostpci0: mapping=RTX_3090_5,pcie=1
machine: pc-q35-8.1
memory: 16384
meta: creation-qemu=8.1.2,ctime=1701005861
name: pve-win-5
net0: virtio=BC:24:11:B1:8A:6E,bridge=vmbr0,firewall=1
numa: 1
onboot: 1
ostype: win11
scsi0: local-lvm:vm-105-disk-1,discard=on,iothread=1,size=128G,ssd=1
scsihw: virtio-scsi-single
smbios1:
sockets: 1
tpmstate0: local-lvm:vm-105-disk-2,size=4M,version=v2.0
vmgenid:

Still got VIDEO_DXGKRNL_FATAL_ERROR BSOD. One way I can trigger it is by telling device manager to scan for hardware changes whenever the application that needs to be able to use WSL is running. Really trippy

Thanks for the suggestion though!

 

[francismunch]

Did you ever figure out what to do to get WSL working under a windows 11 VM? I have a ryzen 7950 and tried i think most everything wtih no luck.