Proxmox GPU Passthrough Error Code 10

Y

yamaharacer

Member
Jul 23, 2023
31
0
6
Hallo, ich versuche nun bereits mit verschiedenen Windows Versionen in einer VM eine GTX 750 ti oder GTX 550 TI durchzureichen. Egal ob in Windows XP, Windows 7 oder Windows 10, ich erhalte immer den Code 10 dass das Gerät nicht gestartet werden konnte. Die Audioschnittstelle scheint zu funktionieren bzw. hier erhalte ich keinen Fehler Code 10. Nur bei der Grafikkarte.

Hardware:
I7 3770
32GB DDR3 1333MHz
1x AMD RX 570
1x GTX 550 TI bzw. GTX 750 TI
Asus P8Z77-V (Bios Einstellung IGPU priorisiert, CSM aktiviert, Intel virtualization aktiviert,, Memory Remap Feature (VT-d) aktiviert)
Samsung Q860 1TB SSD
PVE 8.0

Folgendes habe ich gemacht:

in /etc/default/grub
Code: 
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction video=vesafb:off video=efifb:off video=simplefb:off"
Code: 
update-grub


/etc/modules
Code: 
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd


/etc/modprobe.d/vfio.conf (eine ID die RX die andere die GTX 550 TI)
Code: 
options vfio-pci ids=1002:67df,1002:aaf0,10de:1244,10de:0bee disable_vga=1


/etc/modprobe.d/iommu_unsafe_interrupts.conf
Code: 
options vfio_iommu_type1 allow_unsafe_interrupts=1


/etc/modprobe.d/kvm.conf
Code: 
options kvm ignore_msrs=1
options kvm ignore_msrs=1 report_ignored_msrs=0


/etc/modprobe.d/blacklist.conf
Code: 
blacklist nvidia
blacklist radeon
blacklist nouveau


Code: 
update-initramfs -u


VM Einstellungen ohne UEFI XP

1690115843318.png
1690115876746.png



VM Einstellungen mit UEFI Windows 7
1690115939001.png
1690115965298.png

Weiß jemand weiter und könnte mir eventuell helfen?
 
I wrote this answer in your English thread but that dissapeared...
yamaharacer said:
Hi, I try to passthrough a GTX 550 ti or a GTX 750 ti to a Windows XP VM. Doesnt matter if I passthrough to a XP VM or a Windows 7 VM. Everytime Error Code 10 in device manager for the display driver. A little bit confusing is, that the Audio device hasnt any error code.

Could anybody help me? Dont be confused why I tryed XP and 7. Its because I wanted to know if the problem ist the Uefi setting for passthrough PCIe.


Hardware:

I7 3770

32GB DDR3 1333MHz

1x AMD RX 570

1x GTX 550 TI or GTX 750 TI

Asus P8Z77-V (Bios Settings: IGPU first CSM enabled, Intel virtualization enabled, Memory Remap Feature (VT-d) enabled)

Samsung Q860 1TB SSD

PVE 8.0
Click to expand...
I never managed to passthrough a GTX550 (on a similar system) and switched to AMD (HD7750 at the time). I think other people here have managed passthrough of GTX750 but I have no personal experience with that.
For passthrough of an RX570 you need vendor-reset and it will work fine.
yamaharacer said:
What I have don:

in /etc/default/grub

Code: 
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction video=vesafb:off video=efifb:off video=simplefb:off"

Code: 
update-grub
Click to expand...
video=vesafb:off video=efifb:off video=simplefb:off does nothing any more and you need this work-around if you want to passthrough a GPU that is used during boot.
Please don't use pcie_acs_override. It makes Proxmox ignore the IOMMU boundaries which breaks security but does not guarantee that it will passthrough.
Make sure with cat /proc/cmdline that you are editing the right bootloader.
yamaharacer said:
/etc/modules

Code: 
vfio

vfio_iommu_type1

vfio_pci

vfio_virqfd
Click to expand...
vfio_virqfd no longer exists, you must be using outdated guides.
yamaharacer said:
/etc/modprobe.d/vfio.conf (one ID the RX the other GTX 550 TI)

Code: 
options vfio-pci ids=1002:67df,1002:aaf0,10de:1244,10de:0bee disable_vga=1
Click to expand...
Looks good and early binds the Nvida GPUs to vfio-pci. With vendor-reset you probably don't need the RX570 here.
To make sure vfio-pci loads before nouveau add a softdep nouveau pre: vfio-pci (and similar for the amdgpu driver).
yamaharacer said:
Code: 
blacklist nvidia

blacklist radeon

blacklist nouveau
Click to expand...
I assume nvidia is not even installed and RX570 does not use radeon, so this looks arbitrary and probably wrong.
yamaharacer said:
Code: 
update-initramfs -u

VM settings without UEFI XP

1690115843318.png

1690115876746.png

VM settings with UEFI Windows 7

1690115939001.png

1690115965298.png

Weiß jemand weiter und könnte mir eventuell helfen?
Click to expand...
Sharing the VM configuration files (qm config YOUR_VM_ID) is more complete and easier to read.
Try booting Proxmox with the GTX550 or RX570 (with vendor-reset) if you want to passthrough the GTX750 and/or use the initcall_blacklist=sysfb_init work-around.
Show any message from around the time of starting a VM from journalctl (after making the suggested changes).
You can test basic passthrough with a Ubuntu Live installer ISO. Don't install it, just start your Windows VM with the ISO and see if you get output on a physical display.
 
Thank you for your reply. I will reinstall proxmox from ground, cause I have made many other tryes during this afternoon and will reply tomorrow.
 
yamaharacer said:
Thank you for your reply. I will reinstall proxmox from ground, cause I have made many other tryes during this afternoon and will reply tomorrow.
Click to expand...
If you're restarting, please follow the manual (but you do always need intel_iommu=on) and show your IOMMU groups (without override) with for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done. And let's take it step by step from there (instead of trying everything at once). Are you doing passthrough of the RX570 as well?
 
At the moment I just try to passthrough the Nvidia. Just when this one works well, I want to go on with the RX.

But what I can say at the moment is, that when I startet this task, the RX and the GTX were in the same IOMMU groupe. After pcie_acs_override they were seperated.
 
Last edited:
yamaharacer said:
At the moment I just try to passthrough the Nvidia. Just when this one works well, I want to go on with the RX.
Click to expand...
EDIT: Changed my mind. it will be easier to boot with the RX570, since you can use vendor-reset later to make it work with passthrough anyway. And do the GTX750 first.
 
Last edited:
Yeah like I suspected. But when my plan with the GTX doesnt work I will go on with an other Setup with seperated PCs for each OS. At the moment proxmox would be my prefered solution.
 
leesteken said:
EDIT: Changed my mind. it will be easier to boot with the RX570, since you can use vendor-reset later to make it work with passthrough anyway. And do the GTX750 first.
Click to expand...

So I am at this point of the manual:

dmesg | grep -e DMAR -e IOMMU -e AMD-Vi
Code: 
[    0.011065] ACPI: DMAR 0x000000009C8A2298 0000B8 (v01 INTEL  SNB      00000001 INTL 00000001)
[    0.011079] ACPI: Reserving DMAR table memory at [mem 0x9c8a2298-0x9c8a234f]
[    0.072082] DMAR: IOMMU enabled
[    0.185339] DMAR: Host address width 36
[    0.185340] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[    0.185345] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap c0000020e60262 ecap f0101a
[    0.185347] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[    0.185350] DMAR: dmar1: reg_base_addr fed91000 ver 1:0 cap c9008020660262 ecap f0105a
[    0.185351] DMAR: RMRR base: 0x0000009c814000 end: 0x0000009c825fff
[    0.185353] DMAR: RMRR base: 0x0000009f000000 end: 0x000000af1fffff
[    0.185355] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.185356] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[    0.185357] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.185861] DMAR-IR: Enabled IRQ remapping in x2apic mode
[    0.400561] DMAR: No ATSR found
[    0.400563] DMAR: No SATC found
[    0.400564] DMAR: IOMMU feature pgsel_inv inconsistent
[    0.400566] DMAR: IOMMU feature pass_through inconsistent
[    0.400567] DMAR: dmar0: Using Queued invalidation
[    0.400571] DMAR: dmar1: Using Queued invalidation
[    0.520481] DMAR: Intel(R) Virtualization Technology for Directed I/O
[    3.656812] AMD-Vi: AMD IOMMUv2 functionality not available on this system - This is not a bug.
[    4.228410] i915 0000:00:02.0: [drm] DMAR active, disabling use of stolen memory


for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done
Code: 
IOMMU group 0 00:02.0 VGA compatible controller [0300]: Intel Corporation IvyBridge GT2 [HD Graphics 4000] [8086:0162] (rev 09)
IOMMU group 10 00:1c.4 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge [8086:244e] (rev c4)
IOMMU group 10 05:00.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge [1b21:1080] (rev 03)
IOMMU group 11 00:1c.7 PCI bridge [0604]: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 8 [8086:1e1e] (rev c4)
IOMMU group 12 00:1d.0 USB controller [0c03]: Intel Corporation 7 Series/C216 Chipset Family USB Enhanced Host Controller #1 [8086:1e26] (rev 04)
IOMMU group 13 00:1f.0 ISA bridge [0601]: Intel Corporation Z77 Express Chipset LPC Controller [8086:1e44] (rev 04)
IOMMU group 13 00:1f.2 SATA controller [0106]: Intel Corporation 7 Series/C210 Series Chipset Family 6-port SATA Controller [AHCI mode] [8086:1e02] (rev 04)
IOMMU group 13 00:1f.3 SMBus [0c05]: Intel Corporation 7 Series/C216 Chipset Family SMBus Controller [8086:1e22] (rev 04)
IOMMU group 14 04:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 01)
IOMMU group 15 07:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller [1b21:1042]
IOMMU group 1 00:00.0 Host bridge [0600]: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor DRAM Controller [8086:0150] (rev 09)
IOMMU group 2 00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port [8086:0151] (rev 09)
IOMMU group 2 00:01.1 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port [8086:0155] (rev 09)
IOMMU group 2 01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] [1002:67df] (rev ef)
IOMMU group 2 01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1002:aaf0]
IOMMU group 2 02:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM107 [GeForce GTX 750 Ti] [10de:1380] (rev a2)
IOMMU group 2 02:00.1 Audio device [0403]: NVIDIA Corporation GM107 High Definition Audio Controller [GeForce 940MX] [10de:0fbc] (rev a1)
IOMMU group 3 00:14.0 USB controller [0c03]: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller [8086:1e31] (rev 04)
IOMMU group 4 00:16.0 Communication controller [0780]: Intel Corporation 7 Series/C216 Chipset Family MEI Controller #1 [8086:1e3a] (rev 04)
IOMMU group 5 00:19.0 Ethernet controller [0200]: Intel Corporation 82579V Gigabit Network Connection [8086:1503] (rev 04)
IOMMU group 6 00:1a.0 USB controller [0c03]: Intel Corporation 7 Series/C216 Chipset Family USB Enhanced Host Controller #2 [8086:1e2d] (rev 04)
IOMMU group 7 00:1b.0 Audio device [0403]: Intel Corporation 7 Series/C216 Chipset Family High Definition Audio Controller [8086:1e20] (rev 04)
IOMMU group 8 00:1c.0 PCI bridge [0604]: Intel Corporation 7 Series/C216 Chipset Family PCI Express Root Port 1 [8086:1e10] (rev c4)
IOMMU group 9 00:1c.3 PCI bridge [0604]: Intel Corporation 7 Series/C216 Chipset Family PCI Express Root Port 4 [8086:1e16] (rev c4)


At this point it seems that I have to use:
Code: 
options vfio_iommu_type1 allow_unsafe_interrupts=1
 
yamaharacer said:
for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done
Code: 
IOMMU group 0 00:02.0 VGA compatible controller [0300]: Intel Corporation IvyBridge GT2 [HD Graphics 4000] [8086:0162] (rev 09)
IOMMU group 10 00:1c.4 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge [8086:244e] (rev c4)
IOMMU group 10 05:00.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge [1b21:1080] (rev 03)
IOMMU group 11 00:1c.7 PCI bridge [0604]: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 8 [8086:1e1e] (rev c4)
IOMMU group 12 00:1d.0 USB controller [0c03]: Intel Corporation 7 Series/C216 Chipset Family USB Enhanced Host Controller #1 [8086:1e26] (rev 04)
IOMMU group 13 00:1f.0 ISA bridge [0601]: Intel Corporation Z77 Express Chipset LPC Controller [8086:1e44] (rev 04)
IOMMU group 13 00:1f.2 SATA controller [0106]: Intel Corporation 7 Series/C210 Series Chipset Family 6-port SATA Controller [AHCI mode] [8086:1e02] (rev 04)
IOMMU group 13 00:1f.3 SMBus [0c05]: Intel Corporation 7 Series/C216 Chipset Family SMBus Controller [8086:1e22] (rev 04)
IOMMU group 14 04:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 01)
IOMMU group 15 07:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller [1b21:1042]
IOMMU group 1 00:00.0 Host bridge [0600]: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor DRAM Controller [8086:0150] (rev 09)
IOMMU group 2 00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port [8086:0151] (rev 09)
IOMMU group 2 00:01.1 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port [8086:0155] (rev 09)
IOMMU group 2 01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] [1002:67df] (rev ef)
IOMMU group 2 01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1002:aaf0]
IOMMU group 2 02:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM107 [GeForce GTX 750 Ti] [10de:1380] (rev a2)
IOMMU group 2 02:00.1 Audio device [0403]: NVIDIA Corporation GM107 High Definition Audio Controller [GeForce 940MX] [10de:0fbc] (rev a1)
IOMMU group 3 00:14.0 USB controller [0c03]: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller [8086:1e31] (rev 04)
IOMMU group 4 00:16.0 Communication controller [0780]: Intel Corporation 7 Series/C216 Chipset Family MEI Controller #1 [8086:1e3a] (rev 04)
IOMMU group 5 00:19.0 Ethernet controller [0200]: Intel Corporation 82579V Gigabit Network Connection [8086:1503] (rev 04)
IOMMU group 6 00:1a.0 USB controller [0c03]: Intel Corporation 7 Series/C216 Chipset Family USB Enhanced Host Controller #2 [8086:1e2d] (rev 04)
IOMMU group 7 00:1b.0 Audio device [0403]: Intel Corporation 7 Series/C216 Chipset Family High Definition Audio Controller [8086:1e20] (rev 04)
IOMMU group 8 00:1c.0 PCI bridge [0604]: Intel Corporation 7 Series/C216 Chipset Family PCI Express Root Port 1 [8086:1e10] (rev c4)
IOMMU group 9 00:1c.3 PCI bridge [0604]: Intel Corporation 7 Series/C216 Chipset Family PCI Express Root Port 4 [8086:1e16] (rev c4)
Click to expand...
I see you have Intel integrated graphics. Do you use that to boot the Proxmox system?
The RX570 and the GTX750 are in the same IOMMU group (2). Maybe you can move one of them to another slot and see if you can separate them?
yamaharacer said:
At this point it seems that I have to use:
Code: 
options vfio_iommu_type1 allow_unsafe_interrupts=1
Click to expand...
Why?
 
ok I have puted it in another slot:

Code: 
IOMMU group 0 00:02.0 VGA compatible controller [0300]: Intel Corporation IvyBridge GT2 [HD Graphics 4000] [8086:0162] (rev 09)
IOMMU group 10 00:1c.4 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge [8086:244e] (rev c4)
IOMMU group 10 04:00.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge [1b21:1080] (rev 03)
IOMMU group 11 00:1c.7 PCI bridge [0604]: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 8 [8086:1e1e] (rev c4)
IOMMU group 12 00:1d.0 USB controller [0c03]: Intel Corporation 7 Series/C216 Chipset Family USB Enhanced Host Controller #1 [8086:1e26] (rev 04)
IOMMU group 13 00:1f.0 ISA bridge [0601]: Intel Corporation Z77 Express Chipset LPC Controller [8086:1e44] (rev 04)
IOMMU group 13 00:1f.2 SATA controller [0106]: Intel Corporation 7 Series/C210 Series Chipset Family 6-port SATA Controller [AHCI mode] [8086:1e02] (rev 04)
IOMMU group 13 00:1f.3 SMBus [0c05]: Intel Corporation 7 Series/C216 Chipset Family SMBus Controller [8086:1e22] (rev 04)
IOMMU group 14 02:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM107 [GeForce GTX 750 Ti] [10de:1380] (rev a2)
IOMMU group 14 02:00.1 Audio device [0403]: NVIDIA Corporation GM107 High Definition Audio Controller [GeForce 940MX] [10de:0fbc] (rev a1)
IOMMU group 15 03:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 01)
IOMMU group 16 06:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller [1b21:1042]
IOMMU group 1 00:00.0 Host bridge [0600]: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor DRAM Controller [8086:0150] (rev 09)
IOMMU group 2 00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port [8086:0151] (rev 09)
IOMMU group 2 01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] [1002:67df] (rev ef)
IOMMU group 2 01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1002:aaf0]
IOMMU group 3 00:14.0 USB controller [0c03]: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller [8086:1e31] (rev 04)
IOMMU group 4 00:16.0 Communication controller [0780]: Intel Corporation 7 Series/C216 Chipset Family MEI Controller #1 [8086:1e3a] (rev 04)
IOMMU group 5 00:19.0 Ethernet controller [0200]: Intel Corporation 82579V Gigabit Network Connection [8086:1503] (rev 04)
IOMMU group 6 00:1a.0 USB controller [0c03]: Intel Corporation 7 Series/C216 Chipset Family USB Enhanced Host Controller #2 [8086:1e2d] (rev 04)
IOMMU group 7 00:1b.0 Audio device [0403]: Intel Corporation 7 Series/C216 Chipset Family High Definition Audio Controller [8086:1e20] (rev 04)
IOMMU group 8 00:1c.0 PCI bridge [0604]: Intel Corporation 7 Series/C216 Chipset Family PCI Express Root Port 1 [8086:1e10] (rev c4)
IOMMU group 9 00:1c.3 PCI bridge [0604]: Intel Corporation 7 Series/C216 Chipset Family PCI Express Root Port 4 [8086:1e16] (rev c4)

Yeah I am using iGPU for booting.
 
Last edited:
yamaharacer said:
ok I have puted it in another slot:

Code: 
IOMMU group 0 00:02.0 VGA compatible controller [0300]: Intel Corporation IvyBridge GT2 [HD Graphics 4000] [8086:0162] (rev 09)
IOMMU group 10 00:1c.4 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge [8086:244e] (rev c4)
IOMMU group 10 04:00.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge [1b21:1080] (rev 03)
IOMMU group 11 00:1c.7 PCI bridge [0604]: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 8 [8086:1e1e] (rev c4)
IOMMU group 12 00:1d.0 USB controller [0c03]: Intel Corporation 7 Series/C216 Chipset Family USB Enhanced Host Controller #1 [8086:1e26] (rev 04)
IOMMU group 13 00:1f.0 ISA bridge [0601]: Intel Corporation Z77 Express Chipset LPC Controller [8086:1e44] (rev 04)
IOMMU group 13 00:1f.2 SATA controller [0106]: Intel Corporation 7 Series/C210 Series Chipset Family 6-port SATA Controller [AHCI mode] [8086:1e02] (rev 04)
IOMMU group 13 00:1f.3 SMBus [0c05]: Intel Corporation 7 Series/C216 Chipset Family SMBus Controller [8086:1e22] (rev 04)
IOMMU group 14 02:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM107 [GeForce GTX 750 Ti] [10de:1380] (rev a2)
IOMMU group 14 02:00.1 Audio device [0403]: NVIDIA Corporation GM107 High Definition Audio Controller [GeForce 940MX] [10de:0fbc] (rev a1)
IOMMU group 15 03:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 01)
IOMMU group 16 06:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller [1b21:1042]
IOMMU group 1 00:00.0 Host bridge [0600]: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor DRAM Controller [8086:0150] (rev 09)
IOMMU group 2 00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port [8086:0151] (rev 09)
IOMMU group 2 01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] [1002:67df] (rev ef)
IOMMU group 2 01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1002:aaf0]
IOMMU group 3 00:14.0 USB controller [0c03]: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller [8086:1e31] (rev 04)
IOMMU group 4 00:16.0 Communication controller [0780]: Intel Corporation 7 Series/C216 Chipset Family MEI Controller #1 [8086:1e3a] (rev 04)
IOMMU group 5 00:19.0 Ethernet controller [0200]: Intel Corporation 82579V Gigabit Network Connection [8086:1503] (rev 04)
IOMMU group 6 00:1a.0 USB controller [0c03]: Intel Corporation 7 Series/C216 Chipset Family USB Enhanced Host Controller #2 [8086:1e2d] (rev 04)
IOMMU group 7 00:1b.0 Audio device [0403]: Intel Corporation 7 Series/C216 Chipset Family High Definition Audio Controller [8086:1e20] (rev 04)
IOMMU group 8 00:1c.0 PCI bridge [0604]: Intel Corporation 7 Series/C216 Chipset Family PCI Express Root Port 1 [8086:1e10] (rev c4)
IOMMU group 9 00:1c.3 PCI bridge [0604]: Intel Corporation 7 Series/C216 Chipset Family PCI Express Root Port 4 [8086:1e16] (rev c4)
Click to expand...
Looks good; GTX750 isolated in group 14. Early bind 10de:1380,10de:0fbc to vfio-pci and add softdeps for snd_hda_intel and nouveau. Maybe show the file you created for it in /etc/modprobe.d/ to double check. After that reboot Proxmox and confirm with lspci -nnks 02:00 that vfio-pci is the kernel driver in use.
 
/etc/modprobe.d/vfio.conf
Code: 
options vfio-pci ids=10de:1380,10de:0fbc disable_vga=1 softdep nouveau pre: vfio-pci softdep snd_hda_intel pre: vfio-pci


lspci -nnks 02:00
Code: 
02:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM107 [GeForce GTX 750 Ti] [10de:1380] (rev a2)
        Subsystem: ASUSTeK Computer Inc. GM107 [GeForce GTX 750 Ti] [1043:84bb]
        Kernel driver in use: vfio-pci
        Kernel modules: nvidiafb, nouveau
02:00.1 Audio device [0403]: NVIDIA Corporation GM107 High Definition Audio Controller [GeForce 940MX] [10de:0fbc] (rev a1)
        Subsystem: ASUSTeK Computer Inc. GM107 High Definition Audio Controller [GeForce 940MX] [1043:84bb]
        Kernel driver in use: vfio-pci
        Kernel modules: snd_hda_intel
 
Last edited:
yamaharacer said:
/etc/modprobe.d/vfio.conf
Code: 
options vfio-pci ids=10de:1380,10de:0fbc disable_vga=1 softdep nouveau pre: vfio-pci softdep snd_hda_intel pre: vfio-pci
Click to expand...
I think those need to be on separate lines:
options vfio-pci ids=10de:1380,10de:0fbc disable_vga=1 softdep nouveau pre: vfio-pci softdep snd_hda_intel pre: vfio-pci
yamaharacer said:
lspci -nnks 02:00
Code: 
02:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM107 [GeForce GTX 750 Ti] [10de:1380] (rev a2)
        Subsystem: ASUSTeK Computer Inc. GM107 [GeForce GTX 750 Ti] [1043:84bb]
        Kernel driver in use: vfio-pci
        Kernel modules: nvidiafb, nouveau
02:00.1 Audio device [0403]: NVIDIA Corporation GM107 High Definition Audio Controller [GeForce 940MX] [10de:0fbc] (rev a1)
        Subsystem: ASUSTeK Computer Inc. GM107 High Definition Audio Controller [GeForce 940MX] [1043:84bb]
        Kernel driver in use: vfio-pci
        Kernel modules: snd_hda_intel
Click to expand...
But it appears to work fine (but I would still put the softdeps on separate lines and rerun update-initramfs -u).

Can you create a VM with OVMF and passthrough of 02:00, All Functions, ROM-Bar, PCI Express and Primary GPU? Set Display to none and try booting Ubuntu Live installer ISO.
Do you see some output on the display connected to the GTX750? If not, can you show the journalctl from starting the VM (and share the VM configuration)?
 
yamaharacer said:
Yeah, I can boot Ubuntu and the GTX gives me an output to my second screen.
Click to expand...
GPU passthrough works then. You could try (hard) stopping and restarting the VM to see if the GTX750 resets properly (or good enough).
You might want to use USB passthrough or try passthrough a whole USB controller like 06:00.0 or 00:14.0 or 00:1a.0 (which are all isolated but you'll have to find which ports belong to which controller) to be able to connect a keyboard and mouse to the VM.
I guess you can try installing Windows on that same VM afterwards.
 
I have managed to get USB work within this VM. Only question is: Does it matter if I tick on PCI-Express or not? It works with and without.

It seems that Windows XP doesnt work. I think its because of EFI. At boot up I get some errors:
Code: 
bdsdxe: failed to load boot0003 "UEFI QEMU HARDDISK QM0001" from PciRoot(0x0)/Pci (0x1F, 0x2)/Sata (0x0, 0xFFFF, 0x0): not found
bdsdxe: failed to load boot0001 "UEFI QEMU HARDDISK QM0003" from PciRoot(0x0)/Pci (0x1F, 0x2)/Sata (0x0, 0xFFFF, 0x0): not found

at the end it ends in EFI Shell

edit:
I have recognized that I have to setup the drives in a sequence. First I have to setup the EFI drive then the rest. After that the errors written were gone and it only ends up in efi shell.
So the problem seems to be that Windows XP has no efi boot. Windows 7 boots without any problems.

So I will go on with setup the RX 570. What do I have to do for this?
 
Last edited:
yamaharacer said:
I have managed to get USB work within this VM. Only question is: Does it matter if I tick on PCI-Express or not? It works with and without.
Click to expand...
It does not really matter.
yamaharacer said:
It seems that Windows XP doesnt work. I think its because of EFI. At boot up I get some errors:
Click to expand...
Can't help with that sorry, just make sure it has no internet connection (as it gets no updates).
yamaharacer said:
So I will go on with setup the RX 570. What do I have to do for this?
Click to expand...
First you need to install vendor-reset first (and make sure it is loaded and activate it for the RX570 each reboot of Proxmox). Here is an old guide and several threads on the English forum.
Same steps as the GTX750 except don't use Primary GPU. You don't really have to early bind it to vfio-pci currently, but it make the process a little cleaner and more future proof.
 
Last edited:
Thank you it seems the link to the old guide isnt correct.

Which commands do I have to use to check if the kernel options are enabled? Grep? Sorry my terminal knowledge is at a beginning stage.
 
You must log in or register to reply here.
Top Bottom