[SOLVED] Win10 VM showing error 43 for Quadro 4000

[bvrulez]

I migrated my proxmox VMs to new hardware, a Dell Workstation with Xeon E5-2683 v3 on 2 sockets (14 cores each).

So, I have kernel 5.15.74-1-pve with proxmox 7.3-3.

I have a Windows 10 VM I backuped and restored. I added the PCI card for the "new" Quadro 4000 to the vm.

In this VM I can see the Quadro 4000 in the device manager. It shows error 43. The device is not shown in Task Manager.

I went through the process of setting up PCI passthrough before successfully for other cards, but I am not sure now.

I tried to add stuff to modprobe.d to make sure proxmos is not using the PCI card for its own vga but when I did it the boot process halted in a way where I did not get a ping from the proxmox back. But the attached monitor went off. I had to boot to recovery mode and undo the modprobe options.

What I added was

options vfio_iommu_type1 allow_unsafe_interrupts=1
options kvm ignore_msrs=1
blacklist radeon
options vfio-pci ids=10de:11fa, 10de:0e0b disable_vga=1

Not sure this is correct because I took it from a video for an AMD host.

My config is attached in pictures.

What did I miss?

 

[leesteken]

If the GPU is used during boot, them you probably need this work-around.

 

[bvrulez]

To be exact, it is a Quadro K4000.

Well, maybe I just need the driver from NVIDIA?

https://www.nvidia.de/content/Drive...nternational-dch-whql.exe&lang=de&type=Quadro

 

[bvrulez]

If the GPU is used during boot, them you probably need this work-around.

Thanks for the fast reply. If I can chose the PCI card to use it as hardware for the VM in the proxmox configuration, does this mean I might not need to disable it for proxmox itself? Or is it normal that I can chose the card for a VM which is actually at that time used by the host?

 

[leesteken]

Or is it normal that I can chose the card for a VM which is actually at that time used by the host?

Proxmox does not manage the details and assumes you know best. If you select a PCI(e) device for passthrough, then when the VM is started VFIO takes ownership of the device. Not all devices reset properly and since recent kernels, the Proxmox/BIOS/UEFI console/display driver does not always properly release all resources in use by the GPU. Please try the work-around and see if it helps but maybe you need other work-arounds as well depending on your GPU.

EDIT: I have no experience with NVidia nor Windows, so you'll need help from other people here as well.

 

[bvrulez]

Adding initcall_blacklist=sysfb_init in /etc/default/grub solved the issue. I did not have to do any extra stuff in modprobe.d. My card is now shown in Task Manager.

 

[RajatSharma]

I migrated my proxmox VMs to new hardware, a Dell Workstation with Xeon E5-2683 v3 on 2 sockets (14 cores each).

So, I have kernel 5.15.74-1-pve with proxmox 7.3-3.

I have a Windows 10 VM I backuped and restored. I added the PCI card for the "new" Quadro 4000 to the vm.

In this VM I can see the Quadro 4000 in the device manager. It shows error 43. The device is not shown in Task Manager.

I went through the process of setting up PCI passthrough before successfully for other cards, but I am not sure now.

I tried to add stuff to modprobe.d to make sure proxmos is not using the PCI card for its own vga but when I did it the boot process halted in a way where I did not get a ping from the proxmox back. But the attached monitor went off. I had to boot to recovery mode and undo the modprobe options.

What I added was

options vfio_iommu_type1 allow_unsafe_interrupts=1
options kvm ignore_msrs=1
blacklist radeon
options vfio-pci ids=10de:11fa, 10de:0e0b disable_vga=1

Not sure this is correct because I took it from a video for an AMD host.

My config is attached in pictures.

What did I miss?

Enable IOMMU in the proxmox install and ensure the BIOS settings of the host, it should allow passthrough

 

[bvrulez]

I installed Davinci Resolve to check if the card is recognised correctly and it seems to be.

I just have one minor issue: I was hoping to use the VM for development in Android Studio where I use a Software Emulator to emulate an Android device. This works really fast on native machines like a three years old Dell Precision Laptop with dedicated GPU. However, in the VM it always feels slow and laggy and I wonder if there is something else missing. Also, the GPU does not have a lot of load. So maybe the load does not get shifted to the passed GPU.

 

[leesteken]

However, in the VM it always feels slow and laggy and I wonder if there is something else missing. Also, the GPU does not have a lot of load.

Could be anything really. Windows is often slow because it does not use para-virtualized drivers/devices automatically like VirtIO on Linux. Did you install VirtIO drivers on Windows?
If you are not watching your VM via a physical display connected to the GPU, maybe it is using software rendering.
Maybe you gave all cores and almost all memory the the VM, which can increase latency and stuttering by almost any other activity on the system. Make sure to enable NUMA and also use two sockets for each VM on a two socket physical machine.
Search the forum for performance tips for Windows VMs.

 

[bvrulez]

Thank you for your answer. Just a quick heads-up: Yes, I installed VirtIO-drivers, I think. No, I did not assign all cores/memory. Yes I use two sockets. Not sure about the software rendering. When I find the time I will come back to it.