Updating W11 vm to 24H2 BSOD

G

Graxo

New Member
Jun 23, 2024
24
0
1
Hi,

I have been using a Windows 11 23H2 development server at home for a while now. This is a virtual machine on Proxmox.
To speed up the machine a bit i use a GPU for this machine, a Nvidia Tesla T4 with a framebuffer of 2GB. So `nvidia-231`.

For the CPU of this machine i always used `host` to allow the use of WSL on this machine. This all worked great.
Now i updated to 24H2 and my machine doesn't boot anymore. It gives a blue screen during startup.

When i change the CPU type to `x86-64-v2-AES` it boots fine, but WSL doesn't work anymore, i think because of nested virtualization?
I would like to use my development machine again and receive updates on it, for now i reverted back to a snapshot of 23H2 and it will work for now but 23H2 will be unsupported at some point. Is there anyone else experiencing the same issues? I will post some hardware specs below.

Proxmox server:

Code: 
CPU: AMD Ryzen 5 4600G
Motherboard: B550-A PRO
RAM: 4x DIMM DDR4 Synchronous Unbuffered (Unregistered) 3200 MHz 32GB (something from Corsair)


Development vm:
OS: Windows 11 23H2 (wanna goto 24H2)
Code: 
agent: 1
bios: ovmf
boot: order=scsi0;ide0;net0
cores: 8
cpu: x86-64-v2-AES (when using host, it gives a BSOD)
efidisk0: local-lvm:vm-314-disk-1,efitype=4m,pre-enrolled-keys=1,size=4M
hostpci0: mapping=TeslaT4,mdev=nvidia-231,pcie=1
machine: pc-q35-9.0
memory: 8192
meta: creation-qemu=9.0.2,ctime=1743501393
name: DevBak
net0: rtl8139=BC:24:11:EF:EB:1A,bridge=vmbr1,tag=130
numa: 0
ostype: win11
scsi0: SSD:vm-314-disk-0,iothread=1,size=150G
scsihw: virtio-scsi-single
smbios1: uuid=fd0e7760-c307-4b5d-ad0a-c6138f73e60c
sockets: 1
tpmstate0: local-lvm:vm-314-disk-2,size=4M,version=v2.0
vmgenid: 96a48486-9d60-430a-acc7-9ea2146cb444

GPU Drivers:
Code: 
GPU Host: Nvidia vGPU v18.1
GPU Guest: Nvidia v572.83

Proxmox version and packages:
Code: 
root@pve:~# pveversion -v
proxmox-ve: 8.4.0 (running kernel: 6.8.12-10-pve)
pve-manager: 8.4.1 (running version: 8.4.1/2a5fa54a8503f96d)
proxmox-kernel-helper: 8.1.1
proxmox-kernel-6.8.12-10-pve-signed: 6.8.12-10
proxmox-kernel-6.8: 6.8.12-10
proxmox-kernel-6.8.12-9-pve-signed: 6.8.12-9
proxmox-kernel-6.8.12-8-pve-signed: 6.8.12-8
proxmox-kernel-6.5.13-6-pve-signed: 6.5.13-6
proxmox-kernel-6.5: 6.5.13-6
ceph-fuse: 17.2.7-pve1
corosync: 3.1.9-pve1
criu: 3.17.1-2+deb12u1
glusterfs-client: 10.3-5
ifupdown2: 3.2.0-1+pmx11
ksm-control-daemon: 1.5-1
libjs-extjs: 7.0.0-5
libknet1: 1.30-pve2
libproxmox-acme-perl: 1.6.0
libproxmox-backup-qemu0: 1.5.1
libproxmox-rs-perl: 0.3.5
libpve-access-control: 8.2.2
libpve-apiclient-perl: 3.3.2
libpve-cluster-api-perl: 8.1.0
libpve-cluster-perl: 8.1.0
libpve-common-perl: 8.3.1
libpve-guest-common-perl: 5.2.2
libpve-http-server-perl: 5.2.2
libpve-network-perl: 0.11.2
libpve-rs-perl: 0.9.4
libpve-storage-perl: 8.3.6
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 6.0.0-1
lxcfs: 6.0.0-pve2
novnc-pve: 1.6.0-2
proxmox-backup-client: 3.4.1-1
proxmox-backup-file-restore: 3.4.1-1
proxmox-firewall: 0.7.1
proxmox-kernel-helper: 8.1.1
proxmox-mail-forward: 0.3.2
proxmox-mini-journalreader: 1.4.0
proxmox-offline-mirror-helper: 0.6.7
proxmox-widget-toolkit: 4.3.10
pve-cluster: 8.1.0
pve-container: 5.2.6
pve-docs: 8.4.0
pve-edk2-firmware: 4.2025.02-3
pve-esxi-import-tools: 0.7.4
pve-firewall: 5.1.1
pve-firmware: 3.15-3
pve-ha-manager: 4.0.7
pve-i18n: 3.4.2
pve-qemu-kvm: 9.2.0-5
pve-xtermjs: 5.5.0-2
qemu-server: 8.3.12
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.2.7-pve2
 
Last edited:
Some questions:
  • What error message/code do you see when you get a BSOD?
  • Which version of the VirtIO drivers are you using? I'm not sure if this is relevant, since you mention that changing the CPU type helps, but depending on the BSOD error, it might be useful to know.
 
l.leahu-vladucu said:
Some questions:
  • What error message/code do you see when you get a BSOD?
  • Which version of the VirtIO drivers are you using? I'm not sure if this is relevant, since you mention that changing the CPU type helps, but depending on the BSOD error, it might be useful to know.
Click to expand...
Thanks for the reply,
I get a `BAD_POOL_HEADER` message in the BSOD and i use version virtio 1.266 version.
 
_gabriel said:
what about host without gpu pass ?
what about a 24H2 clean install in new VM then with gpu pass ?
Click to expand...
Without a GPU it boots, but i want to use a GPU. There was a noticeable performance difference.
I could try a clean install, but i dont really wanna do that. An upgrade should just work. I know 24H2 is like the worst upgrade MS did.. but reinstalling my development machine just because MS f'ed up (again).. just makes me quit Windows in general and find a suitable Linux Distro to work on.
 
Graxo said:
When i change the CPU type to `x86-64-v2-AES` it boots fine, but WSL doesn't work anymore, i think because of nested virtualization?
Click to expand...
Are you using nested virtualization? As mentioned in the Nvidia vGPU documentation, using Nvidia vGPU and Nested Virtualization at the same time is not possible:
NVIDIA vGPU deployments do not support nested virtualization, that is, running a hypervisor in a guest VM. For example, enabling the Hyper-V role in a guest VM running the Windows Server OS is not supported because it entails enabling nested virtualization. Similarly, enabling Windows Hypervisor Platform is not supported because it requires the Hyper-V role to be enabled.
Click to expand...
Of course, changing the CPU type to something else than host will turn off nested virtualization. However, there are some alternatives:
  1. Use a CPU type that matches your hardware (see QEMU CPU types, Intel CPU types, AMD CPU types).
  2. Create a custom CPU type.
  3. Use the host CPU type but disable vmx (for Intel CPUs) or svm (for AMD CPUs). There is an example for vmx in the QEMU documentation.
_gabriel said:
From the docs https://pve.proxmox.com/wiki/Windows_11_guest_best_practices
If you are using a GPU via PCI(e) passthrough, you might need to add
args: -cpu host,kvm=off
Click to expand...
You can also try this and see if it helps. However, keep in mind that this option is not meant to disable KVM, but rather to hide the fact that the OS is running in a VM, while still offering all virtualization features. Some Nvidia drivers had issues when they detected that the driver is running in a VM, which is why it was necessary when using PCI(e) passthrough. If the issue is really related to the availability of nested virtualization, I think setting kvm=off will probably not help, since nested virtualization should still work. It can't hurt to try it out, though :)
 
  • Like
Reactions: _gabriel
You must log in or register to reply here.
Top Bottom