Nvidia GPU passthrough code 43

unf0rg0tt3n

Member
Sep 28, 2016
15
0
6
26
Hey guys,

I have followed a guide (this one) on how to passthrough a GPU.
Everything okay, but in the win10 guest I have a code 43 on the gpu.

I have two Nvidia Quadro600's running in the system.
My qemu config:

Code:
bios: ovmf
bootdisk: ide0
cores: 4
cpu: host,hidden=1
efidisk0: pool2:vm-100-disk-0,size=128K
hostpci0: 04:00,pcie=1
ide0: local-lvm:vm-100-disk-0,size=64G
ide2: local:iso/virtio.iso,media=cdrom,size=363020K
machine: q35
memory: 8192
name: test
net0: e1000=46:30:93:CA:7E:9F,bridge=vmbr0,firewall=1
numa: 0
ostype: win10
parent: faack
scsihw: virtio-scsi-pci
smbios1: uuid=75401fc6-6aca-4d6c-9145-3c182aa6ff24
sockets: 1
vmgenid: bacb3728-24f2-4151-aa3f-3ed45c9aa11c
quadro600_prob.JPG

Anyone a idea on how to troubleshoot or fix this problem?

Thanks,
Dennis
 

tinfever

New Member
Jun 30, 2019
6
0
1
28
I haven't worked with Quadro cards but I had to set the CPU flag for hv_vendor_id=whatever as mentioned in this thread https://forum.proxmox.com/threads/modify-cpu-parameters.38686/.

Here is the line I have used before in the config file. I include a lot of the default flags from 'qm showcmd yourvm#' since I believe I read that the latter -cpu args command will overwrite the default one that Proxmox uses.

args: -cpu 'host,+pdpe1gb,+kvm_pv_unhalt,+kvm_pv_eoi,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_reset,hv_vpindex,hv_runtime,hv_relaxed,hv_synic,hv_stimer,hv_tlbflush,hv_ipi,kvm=off,hv_vendor_id=whatever'
 

unf0rg0tt3n

Member
Sep 28, 2016
15
0
6
26
I haven't worked with Quadro cards but I had to set the CPU flag for hv_vendor_id=whatever as mentioned in this thread https://forum.proxmox.com/threads/modify-cpu-parameters.38686/.

Here is the line I have used before in the config file. I include a lot of the default flags from 'qm showcmd yourvm#' since I believe I read that the latter -cpu args command will overwrite the default one that Proxmox uses.

args: -cpu 'host,+pdpe1gb,+kvm_pv_unhalt,+kvm_pv_eoi,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_reset,hv_vpindex,hv_runtime,hv_relaxed,hv_synic,hv_stimer,hv_tlbflush,hv_ipi,kvm=off,hv_vendor_id=whatever'
Thanks! How to get the vendor id?
 

unf0rg0tt3n

Member
Sep 28, 2016
15
0
6
26
Adding the commands to 100.conf and reinstalling Windows10 actually worked for a bit.
Device manager popped: "success"!

Ofc Windows needed to reboot and the following occurs:

quadro600_prob2.JPG

And.... bootloop.
My config after adding the sugested:

Code:
args: -cpu 'host,+pdpe1gb,+kvm_pv_unhalt,+kvm_pv_eoi,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_reset,hv_vpindex,hv_runtime,hv_relaxed,hv_synic,hv_stimer,hv_tlbflush,hv_ipi,kvm=off,hv_vendor_id=Nvidia43FIX'
bios: ovmf
bootdisk: ide0
cores: 4
cpu: host,hidden=1
efidisk0: pool2:vm-100-disk-0,size=128K
hostpci0: 04:00,pcie=1,x-vga=1
ide0: local-lvm:vm-100-disk-0,size=64G
ide2: local:iso/Windows.iso,media=cdrom,size=4023680K
machine: pc-q35-3.1
memory: 8192
name: test
net0: e1000=46:30:93:CA:7E:9F,bridge=vmbr0,firewall=1
numa: 0
ostype: win10
scsihw: virtio-scsi-pci
smbios1: uuid=75401fc6-6aca-4d6c-9145-3c182aa6ff24
sockets: 1
vga: std
vmgenid: bacb3728-24f2-4151-aa3f-3ed45c9aa11c
 

tinfever

New Member
Jun 30, 2019
6
0
1
28
Check what "qm showcmd 100" gives you regarding cpu flags. Make sure you aren't specifying any flags in your manually added args line that aren't normally included by proxmox defaults. Different CPUs have different flags so if you call one that doesn't work with yours, it wouldn't surprise me if strange things happened.

Also, the 'hv_vendor_id=whatever' was literal. It doesn't matter what is that field as long as it isn't something Nvidia has hard coded their drivers to block.

I'm also not sure how the pc-q35-3.1 and x-vga=1 are going to change things since I haven't worked with those.

Ideally I'd suggest just getting your "qm showcmd 100" and copying the -cpu settings, adding the key 'kvm=off,hv_vendor_id=whatever' bits to the end, and then putting that whole thing in your -args command.

Also, there is a chance that Quadro cards won't work the same way. I've only worked with GTX cards in virtualization.

Here is my entire working config:

Code:
agent: 1
args: -cpu 'host,+pdpe1gb,+kvm_pv_unhalt,+kvm_pv_eoi,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_reset,hv_vpindex,hv_runtime,hv_relaxed,hv_synic,hv_stimer,kvm=off,hv_vendor_id=whatever' -machine q35,kernel_irqchip=on
balloon: 0
bios: ovmf
bootdisk: scsi0
cores: 4
cpu: host,hidden=1
efidisk0: local-zfs:vm-101-disk-1,size=128K
hostpci0: 04:00,pcie=1
hugepages: 2
ide0: local:iso/virtio-win-0.1.141.iso,media=cdrom,size=309208K
machine: q35
memory: 204800
name: [redacted]
net0: virtio=[redacted],bridge=vmbr0,firewall=1
numa: 1
ostype: win10
parent: [redacted]
runningmachine: pc-q35-3.0
scsi0: local-zfs:vm-101-disk-0,size=256G
scsihw: virtio-scsi-pci
smbios1: uuid=[redacted]
sockets: 2
vga: std
vmgenid: [redacted]
 

elurex

Active Member
Oct 28, 2015
174
4
38
Taiwan
I am using proxmox ve 6.0 and default its cpu args already have the following

Code:
-cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=proxmox,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_reset,hv_vpindex,hv_runtime,hv_relaxed,hv_synic,hv_stimer,hv_ipi,kvm=off'

following is my vm conf
Code:
agent: 1
bios: ovmf
boot: c
bootdisk: scsi0
cores: 8
cpu: host
efidisk0: rbd:vm-102-disk-0,size=1M
hostpci0: 04:00,pcie=1,x-vga=1
machine: pc-q35-3.1
memory: 16384
name: winclient102
net0: virtio=3A:C2:94:31:A4:C4,bridge=vmbr1
numa: 1
ostype: win10
scsi0: rbd:vm-102-disk-1,cache=writeback,discard=on,size=60G
scsi1: rbd:vm-102-disk-2,cache=writeback,discard=on,size=400G
scsihw: virtio-scsi-pci
smbios1: uuid=c0ea7767-16d4-4c0d-aa48-2b8150176d16
sockets: 1
 

unf0rg0tt3n

Member
Sep 28, 2016
15
0
6
26
I am using proxmox ve 6.0 and default its cpu args already have the following

Code:
-cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=proxmox,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_reset,hv_vpindex,hv_runtime,hv_relaxed,hv_synic,hv_stimer,hv_ipi,kvm=off'

following is my vm conf
Code:
agent: 1
bios: ovmf
boot: c
bootdisk: scsi0
cores: 8
cpu: host
efidisk0: rbd:vm-102-disk-0,size=1M
hostpci0: 04:00,pcie=1,x-vga=1
machine: pc-q35-3.1
memory: 16384
name: winclient102
net0: virtio=3A:C2:94:31:A4:C4,bridge=vmbr1
numa: 1
ostype: win10
scsi0: rbd:vm-102-disk-1,cache=writeback,discard=on,size=60G
scsi1: rbd:vm-102-disk-2,cache=writeback,discard=on,size=400G
scsihw: virtio-scsi-pci
smbios1: uuid=c0ea7767-16d4-4c0d-aa48-2b8150176d16
sockets: 1
So I tried a few of the configurations.
the first one gave:

quadro600_prob3.JPG

After removing the arguments and keep: kvm=off
It did this:

quadro600_prob4.JPG

So when downloading the official driver package it resulted in finding this: https://www.nvidia.com/en-us/data-center/virtual-gpu-technology/
Only for quadro cards, that link was on the download page of the driver.
So maybe nvidia is just preventing me from gpu passthrough because they want me to pay for a license.
 

unf0rg0tt3n

Member
Sep 28, 2016
15
0
6
26
So my suspicion was correct! Quadro doens't like it. When I added the firepro in a seabios VM without any additional config but adding the pci, it booted and patched the vm to my physical screen.
The quadro got recognized tho
quadro600_prob5.JPG
 
Last edited:

dashtesla

New Member
Nov 3, 2019
5
1
3
27
Ja Nederland :D I'm about to do the same with a Quadro P4000 myself so we should probably stay in touch and keep each other updated on test/results.
 
  • Like
Reactions: unf0rg0tt3n

dashtesla

New Member
Nov 3, 2019
5
1
3
27
Also I'm using Windows Server 2019 for my VMs I don't ever use Windows 10 for anything it's too cluttered and full of nonsense, also quadro drivers might work differently on Server 2019 but we'll find out I guess.
 

mbosma

Member
Dec 3, 2018
41
4
8
25

unf0rg0tt3n

Member
Sep 28, 2016
15
0
6
26
Another dutchie here, I've had good experiences with quadro and gtx cards dumping the bios and mounting it to the vm.
Here's a tutorial on the dumping part:
https://forums.unraid.net/topic/41951-gpu-passthrough-with-only-one-card/page/2/?tab=comments#comment-478049
And a snippet from the pve man pages on mounting the dump:
I thought it wasn't possible to mount a .bin file as it's bios file. Thought it had to be a .rom file.
Can I still leave all the settings as they are working now (seabios and other defaults, which let the firepro work.)?
I might try this out later
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!