HBA PCI Passthrough - Failed to set group container: Invalid argume

[rhysmcqueen]

Reinstalled Proxmox on an old host that I am running Truenas as a VM.

After updating the grub to include IOMMU commands:

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt" ```

Ran: Update-grub
rebooted
Added the modules to /etc/modules:

vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

When I add the PCI device to the VM I get this error in Proxmox:

kvm: -device vfio-pci,host=0000:02:00.0,id=hostpci0,bus=pci.0,addr=0x10: vfio 0000:02:00.0: failed to setup container for group 26: Failed to set group container: Invalid argument
TASK ERROR: start failed: QEMU exited with code 1

Here are some other logs that I may find relevant:

root@jj:~# dmesg | grep -e DMAR -e IOMMU
[    0.009998] ACPI: DMAR 0x000000007BAFE000 000140 (v01 DELL   PE_SC3   00000001 DELL 00000001)
[    0.010034] ACPI: Reserving DMAR table memory at [mem 0x7bafe000-0x7bafe13f]
[    0.315590] DMAR: IOMMU enabled
[    0.782432] DMAR: Host address width 46
[    0.782433] DMAR: DRHD base: 0x000000fbffc000 flags: 0x0
[    0.782441] DMAR: dmar0: reg_base_addr fbffc000 ver 1:0 cap d2078c106f0466 ecap f020df
[    0.782444] DMAR: DRHD base: 0x000000c7ffc000 flags: 0x1
[    0.782450] DMAR: dmar1: reg_base_addr c7ffc000 ver 1:0 cap d2078c106f0466 ecap f020df
[    0.782452] DMAR: RMRR base: 0x000000675d6000 end: 0x0000006f5ddfff
[    0.782456] DMAR: RMRR base: 0x0000007ae07000 end: 0x0000007af06fff
[    0.782457] DMAR: ATSR flags: 0x0
[    0.782460] DMAR: ATSR flags: 0x0
[    0.782462] DMAR-IR: IOAPIC id 10 under DRHD base  0xfbffc000 IOMMU 0
[    0.782464] DMAR-IR: IOAPIC id 8 under DRHD base  0xc7ffc000 IOMMU 1
[    0.782466] DMAR-IR: IOAPIC id 9 under DRHD base  0xc7ffc000 IOMMU 1
[    0.782467] DMAR-IR: HPET id 0 under DRHD base 0xc7ffc000
[    0.782469] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit.
[    0.782470] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the BIOS setting.
[    0.783135] DMAR-IR: Enabled IRQ remapping in xapic mode
[    1.288111] DMAR: [Firmware Bug]: RMRR entry for device 02:00.0 is broken - applying workaround
[    1.288169] DMAR: No SATC found
[    1.288171] DMAR: dmar0: Using Queued invalidation
[    1.288175] DMAR: dmar1: Using Queued invalidation
[    1.302627] DMAR: Intel(R) Virtualization Technology for Directed I/O
[  186.437435] vfio-pci 0000:02:00.0: Firmware has requested this device have a 1:1 IOMMU mapping, rejecting configuring the device without a 1:1 mapping. Contact your platform vendor.

The weird part is. I had this host working with the HBA passed through to a VM before I re-installed proxmox....

 

[rhysmcqueen]

I have some more information I found which seems similar.

M

Thread 'Problems with GPU Passthrough since 8.2'

Apr 26, 2024

Hi,

I have had a problem with GPU passthrough since updating to 8.2. The VM to which I have passed the GPU no longer starts and has the following error message:

kvm: -device vfio-pci,host=0000:06:00.0,id=hostpci0.0,bus=ich9-pcie-port-1,addr=0x0.0,x-vga=on,multifunction=on: vfio 0000:06:00.0: failed to setup container for group 20: Failed to set group container: Invalid argument
TASK ERROR: start failed: QEMU exited with code 1

I have these logs on the host:

root@hvs:~# dmesg | grep -e DMAR -e IOMMU -e AMD-Vi
[    0.512344] AMD-Vi: Using global IVHD EFR:0xf77ef22294ada...

• merasil
• Replies: 91
• Forum: Proxmox VE: Installation and configuration

• 

as well as this output:

dmesg | grep -i vfio
[    4.824628] VFIO - User Level meta-driver version: 0.3
[  186.437435] vfio-pci 0000:02:00.0: Firmware has requested this device have a 1:1 IOMMU mapping, rejecting configuring the device without a 1:1 mapping. Contact your platform vendor.
[ 2161.540838] vfio-pci 0000:03:00.0: vgaarb: VGA decodes changed: olddecodes=io+mem,decodes=io+mem:owns=none

And:

grep '' /sys/kernel/iommu_groups/*/reserved_regions
/sys/kernel/iommu_groups/26/reserved_regions:0x00000000675d6000 0x000000006f5ddfff direct