GPU PASSTHROUGH GTX560TI error 43 problem

mercury131

New Member
Aug 7, 2018
23
4
3
29
Hello everyone!
I try to PASSTHROUGH my nvidia GTX560TI in proxmox.

I done this steps:

Change grub parameters to this
Code:
GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on"
add to /etc/modules this lines:
Code:
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
execute this command, because Interrupt remapping not supported on my system
Code:
echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf
I run this command to check iommu group for pci devices
Code:
root@proxmox-lab1:~# find /sys/kernel/iommu_groups/ -type l
/sys/kernel/iommu_groups/7/devices/0000:00:14.1
/sys/kernel/iommu_groups/15/devices/0000:02:00.0
/sys/kernel/iommu_groups/5/devices/0000:00:13.2
/sys/kernel/iommu_groups/5/devices/0000:00:13.0
/sys/kernel/iommu_groups/13/devices/0000:00:16.2
/sys/kernel/iommu_groups/13/devices/0000:00:16.0
/sys/kernel/iommu_groups/3/devices/0000:00:11.0
/sys/kernel/iommu_groups/11/devices/0000:00:14.5
/sys/kernel/iommu_groups/1/devices/0000:00:02.0
/sys/kernel/iommu_groups/8/devices/0000:00:14.2
/sys/kernel/iommu_groups/6/devices/0000:00:14.0
/sys/kernel/iommu_groups/14/devices/0000:01:00.1
/sys/kernel/iommu_groups/14/devices/0000:01:00.0
/sys/kernel/iommu_groups/4/devices/0000:00:12.2
/sys/kernel/iommu_groups/4/devices/0000:00:12.0
/sys/kernel/iommu_groups/12/devices/0000:00:15.0
/sys/kernel/iommu_groups/12/devices/0000:05:00.0
/sys/kernel/iommu_groups/12/devices/0000:04:00.0
/sys/kernel/iommu_groups/12/devices/0000:00:15.3
/sys/kernel/iommu_groups/2/devices/0000:00:09.0
/sys/kernel/iommu_groups/10/devices/0000:00:14.4
/sys/kernel/iommu_groups/0/devices/0000:00:00.0
/sys/kernel/iommu_groups/9/devices/0000:00:14.3
then I find my GPU in lspci output ? in vm config I use
Code:
hostpci0: 01:00.0;01:00.1
Also I add venvor ID of my GPU to file /etc/modprobe.d/vfio.conf
Code:
root@proxmox-lab1:~# cat /etc/modprobe.d/vfio.conf
options vfio-pci ids=10de:1200,10de:0e0c disable_vga=1
root@proxmox-lab1:~#
Then I check GPU bios
Code:
# cd /sys/bus/pci/devices/0000:01:00.0/
# echo 1 > rom
# cat rom > /tmp/image.rom
# echo 0 > rom
Code:
./rom-parser /tmp/image.rom
I dont see type 3 in output
Code:
root@proxmox-lab1:~/rom-parser# ./rom-parser /tmp/image.rom
Valid ROM signature found @0h, PCIR offset 188h
        PCIR: type 0 (x86 PC-AT), vendor: 10de, device: 1200, class: 030000
        PCIR: revision 0, vendor revision: 1
        Last image
And because I have nvidia card, I add this
Code:
echo "options kvm ignore_msrs=1" > /etc/modprobe.d/kvm.conf
And, after all this steps I create new windows 10 VM with OVMF.
VM config:
Code:
balloon: 0
bios: ovmf
bootdisk: sata0
cores: 8
cpu: host,hidden=1
hostpci0: 01:00.0;01:00.1,x-vga=1,pcie=1
ide2: local:iso/Windows10RS4x64.iso,media=cdrom
machine: q35
memory: 8000
name: gpu2
net0: e1000=56:17:1A:02:22:90,bridge=vmbr0
numa: 1
ostype: win8
parent: works
sata0: Gold:vm-107-disk-1,cache=writethrough,size=32G
sata1: Gold:vm-107-disk-2,cache=writethrough,size=100G
scsihw: virtio-scsi-pci
smbios1: uuid=01bcabe6-f275-47d9-86f3-35304a9e5f32
sockets: 1
VM won't boot with ,x-vga=1 parameter.

If I remove this and change line to this
Code:
hostpci0: 01:00.0;01:00.1,pcie=1
VM will boot, I can connect via NoVNC
and see this in device manager - error 43 on NVIDIA GTX 560TI

Nvidia latest drivers installed.

What I do wrong?

My hardware:

MB - asrock 970m pro3
CPU - 8 x AMD FX(tm)-8320 Eight-Core Processor
Software:
Code:
root@proxmox-lab1:~/rom-parser# pveversion --verbose
proxmox-ve: 5.2-2 (running kernel: 4.15.17-1-pve)
pve-manager: 5.2-1 (running version: 5.2-1/0fcd7879)
pve-kernel-4.15: 5.2-1
pve-kernel-4.15.17-1-pve: 4.15.17-9
corosync: 2.4.2-pve5
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.0-8
libpve-apiclient-perl: 2.0-4
libpve-common-perl: 5.0-31
libpve-guest-common-perl: 2.0-16
libpve-http-server-perl: 2.0-8
libpve-storage-perl: 5.0-23
libqb0: 1.0.1-1
lvm2: 2.02.168-pve6
lxc-pve: 3.0.0-3
lxcfs: 3.0.0-1
novnc-pve: 0.6-4
proxmox-widget-toolkit: 1.0-18
pve-cluster: 5.0-27
pve-container: 2.0-23
pve-docs: 5.2-3
pve-firewall: 3.0-8
pve-firmware: 2.0-4
pve-ha-manager: 2.0-5
pve-i18n: 1.0-5
pve-libspice-server1: 0.12.8-3
pve-qemu-kvm: 2.11.1-5
pve-xtermjs: 1.0-5
qemu-server: 5.0-26
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3
zfsutils-linux: 0.7.8-pve1~bpo9
Thanks in advance!
I don't understand where I wrong
 
  • Like
Reactions: AlexLup

mercury131

New Member
Aug 7, 2018
23
4
3
29
I try to use another bios with romfile=vbios.bin option. Nothing change.

I remove parameter x-vga=1 ,after that my vm will work and windows will load.
I remove nvidia driver and my GPU from device manager.
After that I power off my VM and change PCI to back in config
Code:
hostpci0: 01:00.0;01:00.1,x-vga=1,pcie=1
After that I power on VM, and sucessfull connect to VM via RDP.
But if I I install NVIDIA driver and reboot , windows cant boot after install drivers.

I try to move my card to another PCI-E slot and change this slot as primary in BIOS setting - it's not resolve problem - still 43 error.
Also I try to change VGA adapter to my OLD 9500GT and that card works fine.

What I can do to troubleshoot and fix my problem?
 

FastLaneJB

Member
Feb 3, 2012
90
6
8
I think you need the x-vga=1 as that has a fix to solve the 43 error. The 43 error is simply that the Nvidia drivers have detected they are in a VM and you don't have a supported card for that. Only some Quadro cards are officially supported.

However x-vga=1 also disables the KVM VGA card so the console won't work. You'll need to plug a screen into the card direct or RDP to the server. You won't be able to see the screen otherwise.
 
  • Like
Reactions: AlexLup

mercury131

New Member
Aug 7, 2018
23
4
3
29
I think you need the x-vga=1 as that has a fix to solve the 43 error. The 43 error is simply that the Nvidia drivers have detected they are in a VM and you don't have a supported card for that. Only some Quadro cards are officially supported.

However x-vga=1 also disables the KVM VGA card so the console won't work. You'll need to plug a screen into the card direct or RDP to the server. You won't be able to see the screen otherwise.
I try add option x-vga=1 , error 43 doesn't gone =(
I can connect to VM via RDP, but if I install NVIDIA driver my windows will crash after reboot.
If I remove x-vga=1 from config I can boot windows and see error 43 in device manager.

If I passthrough my old NVIDIA 9500GT it works Fine, without error 43.
 

mercury131

New Member
Aug 7, 2018
23
4
3
29
Hi!
I just change my GPU to RX560, and it works fine with same vm and proxmox settings.
I think nvidia driver detect vm and generate error 43.
I already try to patch driver via github utility, IT doesnt help, I got same error 43
 
  • Like
Reactions: AlexLup

kakao73

New Member
Aug 1, 2018
14
0
1
Hi!
I just change my GPU to RX560, and it works fine with same vm and proxmox settings.
I think nvidia driver detect vm and generate error 43.
I already try to patch driver via github utility, IT doesnt help, I got same error 43
I tried RX560 in windows 10 VM but met an error, VIDEO_TDR_FAILURE on atikmpag.sys. How did you do that? Please, could you post your config?
 

mercury131

New Member
Aug 7, 2018
23
4
3
29
I tried RX560 in windows 10 VM but met an error, VIDEO_TDR_FAILURE on atikmpag.sys. How did you do that? Please, could you post your config?
Hello!

I got this settings on my home proxmox lab:
Code:
/etc/default/grub

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="Proxmox Virtual Environment"
GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on"
GRUB_CMDLINE_LINUX="root=ZFS=rpool/ROOT/pve-1 boot=zfs"
Code:
root@proxmox-lab1:~# cat  /etc/modules
# /etc/modules: kernel modules to load at boot time.#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.

vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
I dont create this file
Code:
echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf
Code:
cat /etc/modprobe.d/vfio.conf
options vfio-pci ids=1002:67ff,1002:aae0 disable_vga=1#options vfio-pci ids=1002:67ff,1002:aae0
Code:
cat /etc/modprobe.d/blacklist.conf
blacklist radeon
blacklist nouveau
My Virtual Machine Config:

Code:
bios: ovmfboot: dcn
bootdisk: sata0
cores: 8
cpu: host,hidden=1
efidisk0: Gold:vm-108-disk-2,size=128K
hostpci0: 01:00.0;01:00.1,pcie=1,x-vga=on
ide2: local:iso/virtio-win-0.1.141.iso,media=cdrom,size=309208K
machine: q35
memory: 4096
name: steam
net0: virtio=32:0A:9B:30:E0:5E,bridge=vmbr0
numa: 0
ostype: win10
sata0: Gold:vm-108-disk-3,size=45G
scsihw: virtio-scsi-pci
smbios1: uuid=16fac416-c8b1-48f8-992e-07a079a9ad48
sockets: 1
vga: std
virtio1: Silver:vm-108-disk-1,cache=writeback,size=500G
I Use Windows 10 RS4 Pro with UEFI , With AMD Driver.

I Use This VM for gaming via Steam Translation and this works fine.
My Videocard - MSI AMD Radeon RX 560 , Radeon RX 560 AERO ITX 4G OC, 4GB, GDDR5, OC, Ret
My CPU - AMD FX 8320, SocketAM3+ OEM
Motherboard ASROCK 970M Pro3, SocketAM3+, AMD 970, mATX, Ret

I hope this information will help you.
 

kakao73

New Member
Aug 1, 2018
14
0
1
Hello!

I Use Windows 10 RS4 Pro with UEFI , With AMD Driver.

I Use This VM for gaming via Steam Translation and this works fine.
My Videocard - MSI AMD Radeon RX 560 , Radeon RX 560 AERO ITX 4G OC, 4GB, GDDR5, OC, Ret
My CPU - AMD FX 8320, SocketAM3+ OEM
Motherboard ASROCK 970M Pro3, SocketAM3+, AMD 970, mATX, Ret

I hope this information will help you.
Thank you for very detail information!!! That is similar to mine but a VM disk type(I tried scsi and block), CPU related parts of host(to me, Intel) and Proxmox filesystem(me, ext4, LVM). I'll do a new trial based your information a few days later.

Yes, already I found it. I doubt whether it does actually need. But really thanks for suggestions. I would try if no alternates exist.
 

mercury131

New Member
Aug 7, 2018
23
4
3
29
Thank you for very detail information!!! That is similar to mine but a VM disk type(I tried scsi and block), CPU related parts of host(to me, Intel) and Proxmox filesystem(me, ext4, LVM). I'll do a new trial based your information a few days later.


Yes, already I found it. I doubt whether it does actually need. But really thanks for suggestions. I would try if no alternates exist.
Also you can try to Upgrade your MotherBoard firmware and check IOMMU option is enabled in your BIOS
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!