GPU Passthrough

[TechnicalPeasant]

I wanna try that.

My VM wont start if I select my nvidia GPU.

Here is the error:

swtpm_setup: Not overwriting existing state file.
kvm: -device vfio-pci,host=0000:07:00.0,id=hostpci0.0,bus=ich9-pcie-port-1,addr=0x0.0,multifunction=on: vfio 0000:07:00.0: failed to setup container for group 35: No available IOMMU models
stopping swtpm instance (pid 365327) due to QEMU startup error
TASK ERROR: start failed: QEMU exited with code 1

dmesg | grep -e DMAR -e IOMMU
[ 0.008813] ACPI: DMAR 0x00000000BDDAD200 00060C (v01 HP ProLiant 00000001 \xd2? 0000162E)
[ 0.008882] ACPI: Reserving DMAR table memory at [mem 0xbddad200-0xbddad80b]
[ 0.336976] DMAR: IOMMU enabled
[ 0.806763] DMAR: Host address width 46
[ 0.806765] DMAR: DRHD base: 0x000000fbefe000 flags: 0x0
[ 0.806772] DMAR: dmar0: reg_base_addr fbefe000 ver 1:0 cap d2078c106f0466 ecap f020de
[ 0.806775] DMAR: DRHD base: 0x000000dbffe000 flags: 0x1
[ 0.806779] DMAR: dmar1: reg_base_addr dbffe000 ver 1:0 cap d2078c106f0466 ecap f020de
[ 0.806781] DMAR: RMRR base: 0x000000bdffd000 end: 0x000000bdffffff
[ 0.806783] DMAR: RMRR base: 0x000000bdff6000 end: 0x000000bdffcfff
[ 0.806785] DMAR: RMRR base: 0x000000bdf83000 end: 0x000000bdf84fff
[ 0.806786] DMAR: RMRR base: 0x000000bdf7f000 end: 0x000000bdf82fff
[ 0.806787] DMAR: RMRR base: 0x000000bdf6f000 end: 0x000000bdf7efff
[ 0.806791] DMAR: RMRR base: 0x000000bdf6e000 end: 0x000000bdf6efff
[ 0.806792] DMAR: RMRR base: 0x000000000f4000 end: 0x000000000f4fff
[ 0.806794] DMAR: RMRR base: 0x000000000e8000 end: 0x000000000e8fff
[ 0.806795] DMAR: [Firmware Bug]: No firmware reserved region can cover this RMRR [0x00000000000e8000-0x00000000000e8fff], contact BIOS vendor for fixes
[ 0.806843] DMAR: [Firmware Bug]: Your BIOS is broken; bad RMRR [0x00000000000e8000-0x00000000000e8fff]
[ 0.806845] DMAR: RMRR base: 0x000000bddde000 end: 0x000000bdddefff
[ 0.806846] DMAR: ATSR flags: 0x0
[ 0.806850] DMAR-IR: IOAPIC id 10 under DRHD base 0xfbefe000 IOMMU 0
[ 0.806852] DMAR-IR: IOAPIC id 8 under DRHD base 0xdbffe000 IOMMU 1
[ 0.806853] DMAR-IR: IOAPIC id 0 under DRHD base 0xdbffe000 IOMMU 1
[ 0.806855] DMAR-IR: HPET id 0 under DRHD base 0xdbffe000
[ 0.806856] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit.
[ 0.806857] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the BIOS setting.
[ 0.807636] DMAR-IR: Enabled IRQ remapping in xapic mode
[ 1.275575] DMAR: No SATC found
[ 1.275580] DMAR: dmar0: Using Queued invalidation
[ 1.275590] DMAR: dmar1: Using Queued invalidation
[ 1.281340] DMAR: Intel(R) Virtualization Technology for Directed I/O


Any idea?

 

[leesteken]

Maybe it's the same problem as this and maybe you can use the same solution? What is the output of cat /proc/cmdline?
Or maybe the module vfio_iommu_type1 is not loaded. What is the outpot of lsmod | grep vfio? Did you follow the steps in the Proxmox manual?

 

[TechnicalPeasant]

Maybe it's the same problem as this and maybe you can use the same solution? What is the output of cat /proc/cmdline?
Or maybe the module vfio_iommu_type1 is not loaded. What is the outpot of lsmod | grep vfio? Did you follow the steps in the Proxmox manual?

cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.15.107-2-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on iommu=pt

lsmod | grep vfio
vfio_pci 16384 0
vfio_pci_core 73728 1 vfio_pci
vfio_virqfd 16384 1 vfio_pci_core
irqbypass 16384 30 vfio_pci_core,kvm
vfio 45056 1 vfio_pci_core

 

[leesteken]

lsmod | grep vfio
vfio_pci 16384 0
vfio_pci_core 73728 1 vfio_pci
vfio_virqfd 16384 1 vfio_pci_core
irqbypass 16384 30 vfio_pci_core,kvm
vfio 45056 1 vfio_pci_core

The module vfio_iommu_type1 is not loaded. This is probably the cause of the No available IOMMU models error. Please check the contents of /etc/modules against the instructions in the manual.

 

[TechnicalPeasant]

The module vfio_iommu_type1 is not loaded. This is probably the cause of the No available IOMMU models error. Please check the contents of /etc/modules against the instructions in the manual.

lsmod | grep vfio, now shows:

vfio_pci 16384 0
vfio_pci_core 73728 1 vfio_pci
vfio_virqfd 16384 1 vfio_pci_core
irqbypass 16384 30 vfio_pci_core,kvm
vfio_iommu_type1 45056 0
vfio 45056 2 vfio_pci_core,vfio_iommu_type1

But still an error message when I launch the VM:

swtpm_setup: Not overwriting existing state file.
kvm: -device vfio-pci,host=0000:07:00.0,id=hostpci0.0,bus=ich9-pcie-port-1,addr=0x0.0,multifunction=on: vfio 0000:07:00.0: failed to setup container for group 35: Failed to set iommu for container: Operation not permitted
stopping swtpm instance (pid 5229) due to QEMU startup error
TASK ERROR: start failed: QEMU exited with code 1

lspci -k shows:

07:00.0 VGA compatible controller: NVIDIA Corporation GP107 [GeForce GTX 1050 Ti] (rev ff)
Kernel driver in use: vfio-pci
Kernel modules: nvidiafb, nouveau
07:00.1 Audio device: NVIDIA Corporation GP107GL High Definition Audio Controller (rev ff)
Kernel driver in use: vfio-pci
Kernel modules: snd_hda_intel

Any idea? Thank you in advance.

 

[leesteken]

kvm: -device vfio-pci,host=0000:07:00.0,id=hostpci0.0,bus=ich9-pcie-port-1,addr=0x0.0,multifunction=on: vfio 0000:07:00.0: failed to setup container for group 35: No available IOMMU models

But still the same error message when I launch the VM:

kvm: -device vfio-pci,host=0000:07:00.0,id=hostpci0.0,bus=ich9-pcie-port-1,addr=0x0.0,multifunction=on: vfio 0000:07:00.0: failed to setup container for group 35: Failed to set iommu for container: Operation not permitted

Actually, the error message changed from No available IOMMU models to Operation not permitted. I don't know how to solve this new issue, sorry.

[ 0.806795] DMAR: [Firmware Bug]: No firmware reserved region can cover this RMRR [0x00000000000e8000-0x00000000000e8fff], contact BIOS vendor for fixes
[ 0.806843] DMAR: [Firmware Bug]: Your BIOS is broken; bad RMRR [0x00000000000e8000-0x00000000000e8fff]
[ 0.806845] DMAR: RMRR base: 0x000000bddde000 end: 0x000000bdddefff

Maybe you have the RMRR problem that prevents passthrough. Maybe this thread can help you?