[SOLVED] NVIDIA GT1030 GPU passthrough help

[reeloop]

Hi,

I need help to get my nvidia GT 1030 to work with Windows 10 VM.

Proxmox Virtual Environment 6.2-15

Hardware:
GPU: NVIDIA GT 1030 Low profile
Mainboard: ASRock H270M-ITX/ac (latest BIOS)

What I´ve done so far:
Followed the guide: https://pve.proxmox.com/wiki/Pci_passthrough (GPU OVMF PCI Express Passthrough) // GPU OVMF PCI Passthrough (recommended also don´t work

My configs:
GRUB

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="Proxmox Virtual Environment"
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"
GRUB_CMDLINE_LINUX=""

cat /etc/modules

vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

/etc/modprobe.d/blacklist.conf

lacklist radeon
blacklist nouveau
blacklist nvidia

/etc/modprobe.d/vfio.conf

ptions vfio-pci 10de:1d01,10de:0fb8

/etc/modprobe.d/iommu_unsafe_interrupts.conf (not sure if this is nessesary)

options vfio_iommu_type1 allow_unsafe_interrupts=1

/etc/pve/qemu-server/103.conf

agent: 1
hostpci0: 01:00,pcie=1,x-vga=on
balloon: 1024
bios: ovmf
boot: order=scsi0
cores: 2
cpu: host,hidden=1,flags=+pcid
efidisk0: SAMSUNG_1TB:vm-103-disk-1,size=4M
machine: q35
memory: 16384
name: Windows10pro
net0: virtio=B2:01:51:A1:25:36,bridge=vmbr0,firewall=1
numa: 0
ostype: win10
scsi0: SAMSUNG_1TB:vm-103-disk-0,cache=writeback,discard=on,size=128G
scsihw: virtio-scsi-pci
smbios1: uuid=a2ad64ac-147e-4c31-8aeb-3f271bafb694
sockets: 1
vmgenid: c4acfbcc-4004-480f-82b9-03f2ff2a37b7

VM-Hardware:


VM-Options:


There is no action when I start the VM with the added PCI-Device. If I remove the PCI-Device the VM starts normally.
I get no errors, also when I start the machine with "qm start 103" I get no output, only lockfile is created.

Some informations:

dmesg | grep 'remapping'

[    0.131309] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.132775] DMAR-IR: Enabled IRQ remapping in x2apic mode

find /sys/kernel/iommu_groups/ -type l

/sys/kernel/iommu_groups/7/devices/0000:00:1c.5
/sys/kernel/iommu_groups/5/devices/0000:00:17.0
/sys/kernel/iommu_groups/3/devices/0000:00:14.2
/sys/kernel/iommu_groups/3/devices/0000:00:14.0
/sys/kernel/iommu_groups/11/devices/0000:03:00.0
/sys/kernel/iommu_groups/1/devices/0000:00:01.0
/sys/kernel/iommu_groups/1/devices/0000:01:00.0
/sys/kernel/iommu_groups/1/devices/0000:01:00.1
/sys/kernel/iommu_groups/8/devices/0000:00:1d.0
/sys/kernel/iommu_groups/6/devices/0000:00:1c.0
/sys/kernel/iommu_groups/4/devices/0000:00:16.0
/sys/kernel/iommu_groups/2/devices/0000:00:02.0
/sys/kernel/iommu_groups/10/devices/0000:00:1f.6
/sys/kernel/iommu_groups/0/devices/0000:00:00.0
/sys/kernel/iommu_groups/9/devices/0000:00:1f.2
/sys/kernel/iommu_groups/9/devices/0000:00:1f.0
/sys/kernel/iommu_groups/9/devices/0000:00:1f.3
/sys/kernel/iommu_groups/9/devices/0000:00:1f.4

dmesg | grep -e DMAR -e IOMMU

[    0.008725] ACPI: DMAR 0x000000006EE7D660 0000A8 (v01 INTEL  KBL      00000001 INTL 00000001)
[    0.063672] DMAR: IOMMU enabled
[    0.131299] DMAR: Host address width 39
[    0.131300] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[    0.131303] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 1c0000c40660462 ecap 19e2ff0505e
[    0.131304] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[    0.131306] DMAR: dmar1: reg_base_addr fed91000 ver 1:0 cap d2008c40660462 ecap f050da
[    0.131307] DMAR: RMRR base: 0x0000006e650000 end: 0x0000006e66ffff
[    0.131307] DMAR: RMRR base: 0x0000006f800000 end: 0x0000007fffffff
[    0.131308] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.131309] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[    0.131309] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.132775] DMAR-IR: Enabled IRQ remapping in x2apic mode
[    0.714469] DMAR: No ATSR found
[    0.714491] DMAR: dmar0: Using Queued invalidation
[    0.714493] DMAR: dmar1: Using Queued invalidation
[    0.734257] DMAR: Intel(R) Virtualization Technology for Directed I/O

lspci -nnk

00:00.0 Host bridge [0600]: Intel Corporation Xeon E3-1200 v6/7th Gen Core Processor Host Bridge/DRAM Registers [8086:590f] (rev 05)

        Subsystem: ASRock Incorporation Xeon E3-1200 v6/7th Gen Core Processor Host Bridge/DRAM Registers [1849:590f]

        Kernel driver in use: skl_uncore

00:01.0 PCI bridge [0604]: Intel Corporation Skylake PCIe Controller (x16) [8086:1901] (rev 05)

        Kernel driver in use: pcieport

00:02.0 VGA compatible controller [0300]: Intel Corporation HD Graphics 610 [8086:5902] (rev 04)

        Subsystem: ASRock Incorporation HD Graphics 610 [1849:5902]

        Kernel driver in use: i915

        Kernel modules: i915

00:14.0 USB controller [0c03]: Intel Corporation 200 Series/Z370 Chipset Family USB 3.0 xHCI Controller [8086:a2af]

        Subsystem: ASRock Incorporation 200 Series/Z370 Chipset Family USB 3.0 xHCI Controller [1849:a2af]

        Kernel driver in use: xhci_hcd

        Kernel modules: xhci_pci

00:14.2 Signal processing controller [1180]: Intel Corporation 200 Series PCH Thermal Subsystem [8086:a2b1]

        Subsystem: ASRock Incorporation 200 Series PCH Thermal Subsystem [1849:a2b1]

00:16.0 Communication controller [0780]: Intel Corporation 200 Series PCH CSME HECI [8086:a2ba]

        Subsystem: ASRock Incorporation 200 Series PCH CSME HECI [1849:a2ba]

        Kernel driver in use: mei_me

        Kernel modules: mei_me

00:17.0 RAID bus controller [0104]: Intel Corporation SATA Controller [RAID mode] [8086:2822]

        Subsystem: ASRock Incorporation SATA Controller [RAID mode] [1849:a282]

        Kernel driver in use: ahci

        Kernel modules: ahci

00:1c.0 PCI bridge [0604]: Intel Corporation 200 Series PCH PCI Express Root Port [8086:a292] (rev f0)

        Kernel driver in use: pcieport

00:1c.5 PCI bridge [0604]: Intel Corporation 200 Series PCH PCI Express Root Port [8086:a295] (rev f0)

        Kernel driver in use: pcieport

00:1d.0 PCI bridge [0604]: Intel Corporation 200 Series PCH PCI Express Root Port [8086:a298] (rev f0)

        Kernel driver in use: pcieport

00:1f.0 ISA bridge [0601]: Intel Corporation 200 Series PCH LPC Controller (H270) [8086:a2c4]

        Subsystem: ASRock Incorporation 200 Series PCH LPC Controller (H270) [1849:a2c4]

00:1f.2 Memory controller [0580]: Intel Corporation 200 Series/Z370 Chipset Family Power Management Controller [8086:a2a1]

        Subsystem: ASRock Incorporation 200 Series/Z370 Chipset Family Power Management Controller [1849:a2a1]

00:1f.3 Audio device [0403]: Intel Corporation 200 Series PCH HD Audio [8086:a2f0]

        Subsystem: ASRock Incorporation 200 Series PCH HD Audio [1849:5892]

        Kernel driver in use: snd_hda_intel

        Kernel modules: snd_hda_intel

00:1f.4 SMBus [0c05]: Intel Corporation 200 Series/Z370 Chipset Family SMBus Controller [8086:a2a3]

        Subsystem: ASRock Incorporation 200 Series/Z370 Chipset Family SMBus Controller [1849:a2a3]

        Kernel driver in use: i801_smbus

        Kernel modules: i2c_i801

00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (2) I219-V [8086:15b8]

        Subsystem: ASRock Incorporation Ethernet Connection (2) I219-V [1849:15b8]

        Kernel driver in use: e1000e

        Kernel modules: e1000e

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP108 [10de:1d01] (rev a1)

        Subsystem: Micro-Star International Co., Ltd. [MSI] GP108 [GeForce GT 1030] [1462:8c98]

        Kernel modules: nvidia

01:00.1 Audio device [0403]: NVIDIA Corporation GP108 High Definition Audio Controller [10de:0fb8] (rev a1)

        Subsystem: Micro-Star International Co., Ltd. [MSI] GP108 High Definition Audio Controller [1462:8c98]

        Kernel driver in use: snd_hda_intel

        Kernel modules: snd_hda_intel

03:00.0 Ethernet controller [0200]: Intel Corporation I211 Gigabit Network Connection [8086:1539] (rev 03)

        Subsystem: ASRock Incorporation I211 Gigabit Network Connection [1849:1539]

        Kernel driver in use: igb

        Kernel modules: igb

qm showcmd 103 --pretty

/usr/bin/kvm \
  -id 103 \
  -name Windows10pro \
  -no-shutdown \
  -chardev 'socket,id=qmp,path=/var/run/qemu-server/103.qmp,server,nowait' \
  -mon 'chardev=qmp,mode=control' \
  -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' \
  -mon 'chardev=qmp-event,mode=control' \
  -pidfile /var/run/qemu-server/103.pid \
  -daemonize \
  -smbios 'type=1,uuid=a2ad64ac-147e-4c31-8aeb-3f271bafb694' \
  -drive 'if=pflash,unit=0,format=raw,readonly,file=/usr/share/pve-edk2-firmware//OVMF_CODE.fd' \
  -drive 'if=pflash,unit=1,format=raw,id=drive-efidisk0,size=131072,file=/dev/SAMSUNG_1TB/vm-103-disk-1' \
  -smp '2,sockets=1,cores=2,maxcpus=2' \
  -nodefaults \
  -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \
  -vga none \
  -nographic \
  -no-hpet \
  -cpu 'host,hv_ipi,hv_relaxed,hv_reset,hv_runtime,hv_spinlocks=0x1fff,hv_stimer,hv_synic,hv_time,hv_vapic,hv_vendor_id=proxmox,hv_vpindex,kvm=off,+kvm_pv_eoi,+kvm_pv_unhalt,+pcid' \
  -m 16384 \
  -readconfig /usr/share/qemu-server/pve-q35-4.0.cfg \
  -device 'vmgenid,guid=c4acfbcc-4004-480f-82b9-03f2ff2a37b7' \
  -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' \
  -device 'vfio-pci,host=0000:01:00.0,id=hostpci0.0,bus=ich9-pcie-port-1,addr=0x0.0,multifunction=on' \
  -device 'vfio-pci,host=0000:01:00.1,id=hostpci0.1,bus=ich9-pcie-port-1,addr=0x0.1' \
  -chardev 'socket,path=/var/run/qemu-server/103.qga,server,nowait,id=qga0' \
  -device 'virtio-serial,id=qga0,bus=pci.0,addr=0x8' \
  -device 'virtserialport,chardev=qga0,name=org.qemu.guest_agent.0' \
  -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' \
  -iscsi 'initiator-name=iqn.1993-08.org.debian:01:f4c0efd095e0' \
  -device 'virtio-scsi-pci,id=scsihw0,bus=pci.0,addr=0x5' \
  -drive 'file=/dev/SAMSUNG_1TB/vm-103-disk-0,if=none,id=drive-scsi0,cache=writeback,discard=on,format=raw,aio=threads,detect-zeroes=unmap' \
  -device 'scsi-hd,bus=scsihw0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,bootindex=100' \
  -netdev 'type=tap,id=net0,ifname=tap103i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' \
  -device 'virtio-net-pci,mac=B2:01:51:A1:25:36,netdev=net0,bus=pci.0,addr=0x12,id=net0' \
  -rtc 'driftfix=slew,base=localtime' \
  -machine 'type=q35+pve0' \
  -global 'kvm-pit.lost_tick_policy=discard'

Also tried the "How to know if a Graphics Card is UEFI (OVMF) compatible" section but "cat rom > /tmp/image.rom" failed with:

cat: rom: Input/output error

----- Solution:
missing: "ids=" in /etc/modprobe.d/vfio.conf