VT-D PCI passthrough issues

totalimpact

Renowned Member
Dec 12, 2010
143
24
83
I have set this up on a couple of servers, but this one is giving me trouble, I am trying to figure out if the hardware is just not going to let it happen, or do I just need to tweek my config a bit more.

I am trying to pass through a 32 bit PCI card, have tried a couple of different random cards with no luck.

I am using a Dell T3500 with Xeon X5520, VT-D enabled in bios, and iommu enabled in grub. The system has 4 PCIe slots, and 2 32bit PCI. 3.10.0-5-pve kernel.

What has me concerned is that it seems everything on the PCIe bus passes through, but the 2 32 bit slots dont. The PCI Bridge is on 0000.001e.xx, but it doesnt get an identity mapping in dmesg:

Code:
dmesg |grep IOMMU[    0.000000] Intel-IOMMU: enabled
[    0.151073] dmar: IOMMU 0: reg_base_addr fedc0000 ver 1:0 cap c90780106f0462 ecap f020f6
[    0.881284] IOMMU 0 0xfedc0000: using Queued invalidation
[    0.881290] IOMMU: Setting RMRR:
[    0.881301] IOMMU: Setting identity map for device 0000:00:1d.0 [0xbfe58000 - 0xbfe6ffff]
[    0.881335] IOMMU: Setting identity map for device 0000:00:1d.1 [0xbfe58000 - 0xbfe6ffff]
[    0.881361] IOMMU: Setting identity map for device 0000:00:1d.2 [0xbfe58000 - 0xbfe6ffff]
[    0.881387] IOMMU: Setting identity map for device 0000:00:1d.7 [0xbfe58000 - 0xbfe6ffff]
[    0.881415] IOMMU: Setting identity map for device 0000:00:1a.0 [0xbfe58000 - 0xbfe6ffff]
[    0.881440] IOMMU: Setting identity map for device 0000:00:1a.1 [0xbfe58000 - 0xbfe6ffff]
[    0.881466] IOMMU: Setting identity map for device 0000:00:1a.2 [0xbfe58000 - 0xbfe6ffff]
[    0.881491] IOMMU: Setting identity map for device 0000:00:1a.7 [0xbfe58000 - 0xbfe6ffff]
[    0.881509] IOMMU: Prepare 0-16MiB unity mapping for LPC
[    0.881518] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]

Here is lspci -nn:
Code:
 06:04.0 Communication controller [0780]: Tiger Jet Network Inc. Tiger3XX Modem/ISDN interface [e159:0001]

I tried this in my /etc/default/grub:
Code:
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on pci-stub.ids=e159:0001"


Here is the whole device tree:
Code:
lspci -tv-+-[0000:3f]-+-00.0  Intel Corporation Xeon 5500/Core i7 QuickPath Architecture Generic Non-Core Registers
 |           +-00.1  Intel Corporation Xeon 5500/Core i7 QuickPath Architecture System Address Decoder
 |           +-02.0  Intel Corporation Xeon 5500/Core i7 QPI Link 0
 |           +-02.1  Intel Corporation Xeon 5500/Core i7 QPI Physical 0
 |           +-02.4  Intel Corporation Xeon 5500/Core i7 QPI Link 1
 |           +-02.5  Intel Corporation Xeon 5500/Core i7 QPI Physical 1
 |           +-03.0  Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller
 |           +-03.1  Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Target Address Decoder
 |           +-03.2  Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller RAS Registers
 |           +-03.4  Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Test Registers
 |           +-04.0  Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Control Registers
 |           +-04.1  Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Address Registers
 |           +-04.2  Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Rank Registers
 |           +-04.3  Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 0 Thermal Control Registers
 |           +-05.0  Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Control Registers
 |           +-05.1  Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Address Registers
 |           +-05.2  Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Rank Registers
 |           +-05.3  Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 1 Thermal Control Registers
 |           +-06.0  Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Control Registers
 |           +-06.1  Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Address Registers
 |           +-06.2  Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Rank Registers
 |           \-06.3  Intel Corporation Xeon 5500/Core i7 Integrated Memory Controller Channel 2 Thermal Control Registers
 \-[0000:00]-+-00.0  Intel Corporation 5520/5500/X58 I/O Hub to ESI Port
             +-01.0-[01]--
             +-03.0-[02]----00.0  NVIDIA Corporation G96GL [Quadro FX 580]
             +-07.0-[03]--
             +-14.0  Intel Corporation 7500/5520/5500/X58 I/O Hub System Management Registers
             +-14.1  Intel Corporation 7500/5520/5500/X58 I/O Hub GPIO and Scratch Pad Registers
             +-14.2  Intel Corporation 7500/5520/5500/X58 I/O Hub Control Status and RAS Registers
             +-1a.0  Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4
             +-1a.1  Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5
             +-1a.2  Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6
             +-1a.7  Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2
             +-1c.0-[04]--
             +-1c.5-[05]----00.0  Broadcom Corporation NetXtreme BCM5761 Gigabit Ethernet PCIe
             +-1d.0  Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1
             +-1d.1  Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2
             +-1d.2  Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3
             +-1d.7  Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1
>>>>>    +-1e.0-[06]----04.0  Tiger Jet Network Inc. Tiger3XX Modem/ISDN interface
             +-1f.0  Intel Corporation 82801JIR (ICH10R) LPC Interface Controller
             +-1f.2  Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller
             \-1f.3  Intel Corporation 82801JI (ICH10 Family) SMBus Controller