GPU Passthrough GTX1050Ti won't work

[tweak]

Hi there,

i started yesterday with proxmox and hoped to get my GTX as easy as with vmware to work.

But since the last 12h i try to get this working. but it stucks everytime on code 43.

I have tried all common guides, used different proxmox versions, but it didn't work anyway.

Any ideas left?

What i already tried:

• uefi/stockbios
• pcie/pci passthrough as shown in the wiki
• vendorcodes...
• q35...
• and so one.

thanks for any help

 

[NeroReflex]

Hello,
Code43 is a real pain. Basically it's just nvidia telling you "go buy a quadro for that!".

I know a workaround involving using Windows to dump the vbios.bin file with gpu-z and a script that edit the vbios file. This solution has worked recently judging from user's reports so you should try to follow that lead. Machine type is right to be set to q35 and ovfm uefi.


I have also found this: github.com/sk1080/nvidia-kvm-patcher maybe you should try that.

 

[illustris]

Did you add

args: -cpu host,kvm=off,hv_vendor_id=AMD

in your VM config?

 

[tweak]

Thanks for your feedbacks, but im not sure where is here the difference? For Esxi i didn't need to made any changes in the gpu bios or something.
Just now i am testing with esxi and only thing i need

hypervisor.cpuid.v0 = "FALSE"

and Error 43 is gone.
It works with windows Stock drivers pretty well!

Also the latest nvidia-drivers seems to work without troubles. currently benchmarking if there is any improvement.

 

[tweak]

Here is the current configuration from pve which i try to get working:

agent: 1
args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=1234567890a4,kvm=off'
bios: ovmf
bootdisk: scsi0
cores: 4
efidisk0: local-lvm:vm-100-disk-1,size=4M
hostpci0: 06:00.0,pcie=1,romfile=GTX1050TI.bin
hostpci1: 06:00.1,pcie=1
ide2: local:iso/virtio-win-0.1.171.iso,media=cdrom,size=363020K
localtime: 1
machine: pc-q35-3.1
memory: 8192
name: WinTest
net0: virtio=66:92:05:58:01:23,bridge=vmbr0,firewall=1
numa: 1
ostype: win10
scsi0: local-lvm:vm-100-disk-0,size=50G
scsihw: virtio-scsi-pci
smbios1: uuid=0a8d1c57-a5cb-49eb-bf73-3819ca5311f7
sockets: 1
vmgenid: 7c07571c-eecd-4297-9443-a8e7c2bf0078

another one i tried:

agent: 1
bios: ovmf
bootdisk: scsi0
cores: 4
cpu: host, hidden=1
efidisk0: local-lvm:vm-100-disk-1,size=4M
hostpci0: 06:00.0,x-vga=1,romfile=GTX1050TI.bin
ide2: local:iso/virtio-win-0.1.171.iso,media=cdrom,size=363020K
localtime: 1
machine: pc-q35-3.1
memory: 8192
name: WinTest
net0: virtio=66:92:05:58:01:23,bridge=vmbr0,firewall=1
numa: 0
ostype: win10
scsi0: local-lvm:vm-100-disk-0,size=50G
scsihw: virtio-scsi-pci
smbios1: uuid=0a8d1c57-a5cb-49eb-bf73-3819ca5311f7
sockets: 1
vmgenid: 7c07571c-eecd-4297-9443-a8e7c2bf0078

Both didn't work.

The output of my iommu groups:

/sys/kernel/iommu_groups/0/devices/0000:00:00.0
/sys/kernel/iommu_groups/10/devices/0000:00:1d.0
/sys/kernel/iommu_groups/11/devices/0000:00:1f.0
/sys/kernel/iommu_groups/11/devices/0000:00:1f.5
/sys/kernel/iommu_groups/12/devices/0000:05:00.0
/sys/kernel/iommu_groups/13/devices/0000:05:00.1
/sys/kernel/iommu_groups/14/devices/0000:06:00.0
/sys/kernel/iommu_groups/15/devices/0000:06:00.1
/sys/kernel/iommu_groups/16/devices/0000:01:00.0
/sys/kernel/iommu_groups/17/devices/0000:01:00.1
/sys/kernel/iommu_groups/18/devices/0000:01:00.2
/sys/kernel/iommu_groups/19/devices/0000:01:00.4
/sys/kernel/iommu_groups/1/devices/0000:00:01.0
/sys/kernel/iommu_groups/20/devices/0000:02:00.0
/sys/kernel/iommu_groups/21/devices/0000:02:00.1
/sys/kernel/iommu_groups/2/devices/0000:00:01.1
/sys/kernel/iommu_groups/3/devices/0000:00:12.0
/sys/kernel/iommu_groups/4/devices/0000:00:14.0
/sys/kernel/iommu_groups/4/devices/0000:00:14.2
/sys/kernel/iommu_groups/5/devices/0000:00:16.0
/sys/kernel/iommu_groups/6/devices/0000:00:17.0
/sys/kernel/iommu_groups/7/devices/0000:00:1b.0
/sys/kernel/iommu_groups/8/devices/0000:00:1c.0
/sys/kernel/iommu_groups/9/devices/0000:00:1c.1

Anybody can help here?

Thanks!

 

[Renn]

I'm experiencing the same issue.

I'm running Proxmox 6.1 with kernel 5.3.10-1-pve.

My config:

agent: 1
args: -cpu 'host,hypervisor=false'
bios: ovmf
boot: cdn
bootdisk: virtio0
cores: 8
efidisk0: local-zfs:vm-2000-disk-1,size=128K
hostpci0: 07:00,pcie=1,romfile=MSI.GTX1070.8192.160608_2.rom,x-vga=1
hostpci1: 08:00,pcie=1
ide2: local:iso/virtio-win-0.1.173.iso,media=cdrom,size=385062K
machine: q35,kernel_irqchip=on
memory: 16051
name: console
net0: virtio=D2:18:56:A0:0A:7D,bridge=vmbr0,firewall=1
numa: 1
ostype: win10
parent: before_fixing_43
protection: 1
scsihw: virtio-scsi-single
smbios1: uuid=60ce364f-f8e1-4f46-830f-c4acdf251c99
sockets: 1
usb0: host=0951:1666,usb3=1
vga: none
virtio0: local-zfs:vm-2000-disk-0,backup=0,iothread=1,size=32G
vmgenid: 15e96951-97e6-442b-9df6-5453a5c0399f

MSI.GTX1070.8192.160608_2.rom is the rom I downloaded from techpowerup and modified with https://github.com/Matoking/NVIDIA-vBIOS-VFIO-Patcher, the process of patching had no issue but it just didn't fix Code 43.

hostpci1 is a NVMe SSD and can be ignored.

Here's the result of qm showcmd:

root@host:~# qm showcmd 2000
/usr/bin/kvm -id 2000 -name console -chardev 'socket,id=qmp,path=/var/run/qemu-server/2000.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' -mon 'chardev=qmp-event,mode=control' -pidfile /var/run/qemu-server/2000.pid -daemonize -smbios 'type=1,uuid=60ce364f-f8e1-4f46-830f-c4acdf251c99' -drive 'if=pflash,unit=0,format=raw,readonly,file=/usr/share/pve-edk2-firmware//OVMF_CODE.fd' -drive 'if=pflash,unit=1,format=raw,id=drive-efidisk0,file=/dev/zvol/rpool/data/vm-2000-disk-1' -smp '8,sockets=1,cores=8,maxcpus=8' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' -vga none -nographic -no-hpet -cpu 'kvm64,+lahf_lm,+sep,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=proxmox,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_reset,hv_vpindex,hv_runtime,hv_relaxed,hv_synic,hv_stimer,hv_ipi,enforce,kvm=off' -m 16051 -object 'memory-backend-ram,id=ram-node0,size=16051M' -numa 'node,nodeid=0,cpus=0-7,memdev=ram-node0' -object 'iothread,id=iothread-virtio0' -device 'vmgenid,guid=15e96951-97e6-442b-9df6-5453a5c0399f' -readconfig /usr/share/qemu-server/pve-q35-4.0.cfg -device 'nec-usb-xhci,id=xhci,bus=pci.1,addr=0x1b' -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' -device 'vfio-pci,host=0000:07:00.0,id=hostpci0.0,bus=ich9-pcie-port-1,addr=0x0.0,multifunction=on,romfile=/usr/share/kvm/MSI.GTX1070.8192.160608_2.rom' -device 'vfio-pci,host=0000:07:00.1,id=hostpci0.1,bus=ich9-pcie-port-1,addr=0x0.1' -device 'vfio-pci,host=0000:08:00.0,id=hostpci1,bus=ich9-pcie-port-2,addr=0x0' -device 'usb-host,bus=xhci.0,vendorid=0x0951,productid=0x1666,id=usb0' -chardev 'socket,path=/var/run/qemu-server/2000.qga,server,nowait,id=qga0' -device 'virtio-serial,id=qga0,bus=pci.0,addr=0x8' -device 'virtserialport,chardev=qga0,name=org.qemu.guest_agent.0' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:fbc0a249f310' -drive 'file=/var/lib/vz/template/iso/virtio-win-0.1.173.iso,if=none,id=drive-ide2,media=cdrom,aio=threads' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200' -drive 'file=/dev/zvol/rpool/data/vm-2000-disk-0,if=none,id=drive-virtio0,format=raw,cache=none,aio=native,detect-zeroes=on' -device 'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,iothread=iothread-virtio0,bootindex=100' -netdev 'type=tap,id=net0,ifname=tap2000i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=D2:18:56:A0:0A:7D,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300' -rtc 'driftfix=slew,base=localtime' -machine 'type=q35,kernel_irqchip=on' -global 'kvm-pit.lost_tick_policy=discard' -cpu 'host,hypervisor=false'

 

[tweak]

Can now confirm that everything is working as desired!

Great performance with pve as you can see in the screenshots below compared to esxi.

The solution for my code 43 was to exchange the gpu, first i tried with a GTX1050 instead the GTX1050ti and now i am happy with a GTX1660ti.

For your information, i only need to setup the following parameters:
cpu: host, hidden=1
hostpci0: 06:00.0,pcie=1,x-vga=1

and blacklist the nvidia driver as already known in the wiki.

ESXI



PVE


I think it would possible to reach higher results, but i currently got some issue with the gpu cooling in my hp dl20 case.

 

[Renn]

Can now confirm that everything is working as desired!

Great performance with pve as you can see in the screenshots below compared to esxi.

The solution for my code 43 was to exchange the gpu, first i tried with a GTX1050 instead the GTX1050ti and now i am happy with a GTX1660ti.

For your information, i only need to setup the following parameters:
cpu: host, hidden=1
hostpci0: 06:00.0,pcie=1,x-vga=1

and blacklist the nvidia driver as already known in the wiki.

ESXI
View attachment 13797


PVE
View attachment 13798

I think it would possible to reach higher results, but i currently got some issue with the gpu cooling in my hp dl20 case.

Can you tell me which version of the nVidia driver you are using? And it would be great if you can give a bit more details on how you change the graphics cards because I have tried switching from RTX 2070 to my old GTX 1070 but got no luck. Thanks!

 

[tweak]

Hi Renn,
please take a look at the Screenshots there you can see the driver-version.
The change of the gpu was quite simple, i ejected the GTX1050ti insert the other GTX1050 and it worked fine. I only need to adapt this entry:

echo "options vfio-pci ids=10de:1381,10de:0fbc disable_vga=1" > /etc/modprobe.d/vfio.conf

It's the sample string from the wiki, you need to adapt this one with your specific parameters (lspci -n -s 0x:00)

 

[tweak]

Back Again,

Made a new setup with my SSD ZFS Raid and can't get the passthrough working again.

Ive got now a completly different error, if i add the gpu to the vm and try to start it ends with timeout. Then i can't try to start the machine again because systemd error...
So the only way out would be to reboot the host, but this stucks at timeout PVE-Guest, DBUS.
And now the only way is to press reset.

When i remove the gpu the vm starts imidetly.
So please tell me what is new in proxmox that ive got so mutch troubles now??