Windows 11 VM with Nvidia Quadro P4000 PCI-e passthrough

Mar 20, 2022
23
8
8
I spent a couple of days getting this all working so I thought I would post here in order to help others who may be struggling.

Most of this is in the docs here, but some things are not obvious: https://pve.proxmox.com/wiki/Pci_passthrough

/etc/modprobe.d/kvm.conf
Code:
options kvm ignore_msrs=1

/etc/default/grub
Code:
GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"
GRUB_CMDLINE_LINUX=""

/etc/modules
Code:
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

/etc/modprobe.d/blacklist.conf
Code:
blacklist radeon
blacklist nouveau
blacklist nvidia
blacklist nvidiafb

I did not need to specify the rom file / video bios, and actually had errors following that procedure.

The relevant part of the vm config. The main thing here is I had to disable "primary gpu" (x-vga) or I would get Code 43. Almost every guide I've seen has said to use `x-vga=on` but this was the cause of Code 43 for me.
Code:
hostpci0: 03:00,pcie=1

I also had to stop using SPLICE display as there were a lot of issues. The issues are not there when the gpu is set to primary gpu; however then I got Code 43. Switching to VirtIO-GPU display resolved the graphical issues, although the console is not quite as nice. Now I just remote desktop for most things, but occossionally you may need the console if you want to tweak anything in GeForce Experience.

Lastly, I connected a cheap GPU (GTX 730) in PCI Slot 2, and set the bios to try to use this GPU first. I may take this out to save power later as I don't think it's actually necessary.

With this setup, I have successfully transcoded video using Plex and successfully streamed the desktop using GeForce Experience and Moonlight. Hopefully this is helpful to someone else.
 
One issue that I am still having is I cannot use GeForce Experience game streaming with a dummy dongle, although it does work if I connect to a physical monitor. I am not sure what is going on here as the driver (Generic PnP Monitor) is installed for the dongle, and I know it works on bare metal Windows. If anyone has any ideas on that, I'd appreciate it!

Also, any thoughts for sound? A sound device is shown when connected to RDP, but not when using VNC console. Not sure about the audio devices in the hardware section as they mention SPICE.