GPU Passthrough - PCI device outside of IOMMU range?

[Katz]

I have been working on getting GPU passthrough working on my Threadripper 1920x with an x399-a motherboard.

I have been having an issue in that I get an error: Cannot open iommu_group: No such file or directory when attempting to start the VM. The interesting thing is that this happens only when the HOSTPCI0 # is outside of a certain range, because the device doesn't show up in the IOMMU groups. It appears to be related to the PCI slot that it's in, and weather there are more than one GPU installed. Typically there are no issues when the HOSTPCI0 is around 0a:00.0; however, the majority of the time it ends up being near 40:00.0...and these devices do not show up in the IOMMU groups.

Am I missing some kind of configuration where I can modify which xx:xx.x value the devices are given, or is there are limit as to what devices are included in IOMMU groups? I ask this because in DMESG I see "Found IOMMU cap 0x40"...which oddly corresponds to where my devices aren't found.

Any ideas?

 

[leesteken]

The only way to change the PCI IDs (xx:xx.x) slightly is to move the PCIe cards to other slots, or add or remove cards. I don't thinks this will fix your problem. It would be weird if the devices are not in any IOMMU group, so hopefully it is just a Proxmox web GUI issue.
Can you show us the IOMMU groups with this command line? for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done