[SOLVED] GPU Passthrough fails, No available IOMMU models

[bhdschmidt]

Hi friends,

I try to get GPU Passthrough running. My system has two GPUs:

1) Intel® HD Graphics 4600 integrated in the CPU i5-4690K, this GPU is used by Proxmox
2) NVIDIA GT 1030 which shall be used in a Ubuntu 20.04 VM

I configured my system according to the two tutorials:

https://pve.proxmox.com/wiki/Pci_passthrough

and

https://www.reddit.com/r/homelab/comments/b5xpua/the_ultimate_beginners_guide_to_gpu_passthrough/

But I always receive the following error, when I add the PCI express device to the VM:

kvm: -device vfio-pci,host=0000:01:00.0,id=hostpci0,bus=pci.0,addr=0x10: vfio 0000:01:00.0: failed to setup container for group 11: No available IOMMU models
TASK ERROR: start failed: QEMU exited with code 1

With the tutorials, I could verify that IOMMU and IOMMU Isolation are enabled.


I tried some variations in the configuration, but I always received the same error.
Do you have any suggestions how I should proceed?

Thanks in advance!

Bernhard

 

[leesteken]

What is the output of cat /proc/cmdline and lsmod | grep vfio and for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done?
Make sure to add all four of
vfio vfio_iommu_type1 vfio_pci vfio_virqfd
to /etc/modules and run update-initramfs -u.
.

 

[bhdschmidt]

Hi,

thank you for your fast reply!

The Output of cat /proc/cmdline is:

BOOT_IMAGE=/boot/vmlinuz-5.15.74-1-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction nofb nomodeset video=vesafb:off,efifb:off

The output of cat /proc/cmdline is:

vfio_pci 16384 0 vfio_pci_core 73728 1 vfio_pci vfio_virqfd 16384 1 vfio_pci_core irqbypass 16384 2 vfio_pci_core,kvm vfio 45056 1 vfio_pci_core

The output of for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done is:

IOMMU group 0 00:00.0 Host bridge [0600]: Intel Corporation 4th Gen Core Processor DRAM Controller [8086:0c00] (rev 06) IOMMU group 10 00:1f.0 ISA bridge [0601]: Intel Corporation Q85 Express LPC Controller [8086:8c4c] (rev 04) IOMMU group 10 00:1f.2 SATA controller [0106]: Intel Corporation 8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode] [8086:8c02] (rev 04) IOMMU group 10 00:1f.3 SMBus [0c05]: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller [8086:8c22] (rev 04) IOMMU group 10 00:1f.6 Signal processing controller [1180]: Intel Corporation 8 Series Chipset Family Thermal Management Controller [8086:8c24] (rev 04) IOMMU group 11 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP108 [GeForce GT 1030] [10de:1d01] (rev a1) IOMMU group 12 01:00.1 Audio device [0403]: NVIDIA Corporation GP108 High Definition Audio Controller [10de:0fb8] (rev a1) IOMMU group 1 00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller [8086:0c01] (rev 06) IOMMU group 2 00:02.0 VGA compatible controller [0300]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller [8086:0412] (rev 06) IOMMU group 3 00:03.0 Audio device [0403]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller [8086:0c0c] (rev 06) IOMMU group 4 00:14.0 USB controller [0c03]: Intel Corporation 8 Series/C220 Series Chipset Family USB xHCI [8086:8c31] (rev 04) IOMMU group 5 00:16.0 Communication controller [0780]: Intel Corporation 8 Series/C220 Series Chipset Family MEI Controller #1 [8086:8c3a] (rev 04) IOMMU group 5 00:16.3 Serial controller [0700]: Intel Corporation 8 Series/C220 Series Chipset Family KT Controller [8086:8c3d] (rev 04) IOMMU group 6 00:19.0 Ethernet controller [0200]: Intel Corporation Ethernet Connection I217-V [8086:153b] (rev 04) IOMMU group 7 00:1a.0 USB controller [0c03]: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #2 [8086:8c2d] (rev 04) IOMMU group 8 00:1b.0 Audio device [0403]: Intel Corporation 8 Series/C220 Series Chipset High Definition Audio Controller [8086:8c20] (rev 04) IOMMU group 9 00:1d.0 USB controller [0c03]: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #1 [8086:8c26] (rev 04)

The output of cat /etc/modules results in:

# /etc/modules: kernel modules to load at boot time. # # This file contains the names of kernel modules that should be loaded # at boot time, one per line. Lines beginning with "#" are ignored. vfio vfio_iommu_type1 vfio_pci vfio_virqfd

I have executed update-initramfs -u multiple times.

Kind regards,
Bernhard

 

[leesteken]

The Output of cat /proc/cmdline is:
BOOT_IMAGE=/boot/vmlinuz-5.15.74-1-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction nofb nomodeset video=vesafb:off,efifb:off

Also video=vesafb:off,efifb:off is wrong and should be video=vesafb:off video=efifb:off but that does nothing since kernel version 5.15 where simplefb is used and you need this work-around for single/boot GPU passthrough. Do you really need nofb and nomodeset? What do you think iommu=pt does?

The output of cat /proc/cmdline is:

Do you mean lsmod | grep vfio?

vfio_pci 16384 0 vfio_pci_core 73728 1 vfio_pci vfio_virqfd 16384 1 vfio_pci_core irqbypass 16384 2 vfio_pci_core,kvm vfio 45056 1 vfio_pci_core

vfio_iommu_type1 is missing. It's not loaded. If it is in /etc/modules then it is not applied or otherwise not working.

The output of for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done is:

The IOMMU group information is useless because you are using the pcie_acs_override.

The output of cat /etc/modules results in:

# /etc/modules: kernel modules to load at boot time. # # This file contains the names of kernel modules that should be loaded # at boot time, one per line. Lines beginning with "#" are ignored. vfio vfio_iommu_type1 vfio_pci vfio_virqfd

vfio_iommu_type1 is present but not actually working (see above). Are there any related error messages in the system log? Did you run update-initramfs -u and reboot?

I have executed update-initramfs -u multiple times.

The changes to /etc/modules are not applied. I don't know why. You need to load vfio_iommu_type1 to fix your error message.

 

[bhdschmidt]

Thank you very much,

according to your advice I changed GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub to

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on initcall_blacklist=sysfb_init".

What do you think iommu=pt does?

I copied the parameter from https://pve.proxmox.com/wiki/Pci_passthrough for the PT Mode. As I said I tried a lot of different configurations.


Also, I revisited /etc/modules and retyped the content. Perhaps, there was I typo. Anyways, lsmod | grep vfio prints now vfio_iommu_type1 and I can start the VM without errors!

Thank you a lot, leesteken!