[SOLVED] PCI Passthrough issues

T

totalimpact

Renowned Member
Dec 12, 2010
143
24
83
I am having some trouble, I am doing this in a lab environment, and previously had it working 100% I took some scratch notes on the setup, and in an emergency my lab system had to be re-purposed for something else (formatted), some months later I am trying to get back to my project and it will not work at all - using the same exact server I started on.

I am trying to get a PCI (not express) card working, it is a Matrox Meteor-II Framegrabber, kind of like a serial video capture card - not a video card.

Now running latest 3.4 PVE, tried both 2.6.32-37 and 3.10.0-8 kernels.

I have tried changing PCI slots. Now I am at a loss. I have seen some posts on needing to pass the entire pci bridge, but having trouble isolating it from other devices on the bridge.

Code: 
#### dmesg | grep -e DMAR -e IOMMU
ACPI: DMAR 00000000bf77e0e0 00120 (v01    AMI  OEMDMAR 00000001 MSFT 00000097)
Intel-IOMMU: enabled
dmar: IOMMU 0: reg_base_addr fbffe000 ver 1:0 cap c90780106f0462 ecap f020f6
IOMMU 0xfbffe000: using Queued invalidation
IOMMU: Setting RMRR:
IOMMU: Setting identity map for device 0000:00:1d.0 [0xbf7ec000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1d.1 [0xbf7ec000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1d.2 [0xbf7ec000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1d.7 [0xbf7ec000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1a.0 [0xbf7ec000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1a.1 [0xbf7ec000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1a.2 [0xbf7ec000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1a.7 [0xbf7ec000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1d.0 [0xe6000 - 0xea000]
IOMMU: Setting identity map for device 0000:00:1d.1 [0xe6000 - 0xea000]
IOMMU: Setting identity map for device 0000:00:1d.2 [0xe6000 - 0xea000]
IOMMU: Setting identity map for device 0000:00:1d.7 [0xe6000 - 0xea000]
IOMMU: Setting identity map for device 0000:00:1a.0 [0xe6000 - 0xea000]
IOMMU: Setting identity map for device 0000:00:1a.1 [0xe6000 - 0xea000]
IOMMU: Setting identity map for device 0000:00:1a.2 [0xe6000 - 0xea000]
IOMMU: Setting identity map for device 0000:00:1a.7 [0xe6000 - 0xea000]
IOMMU: Prepare 0-16MiB unity mapping for LPC
IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0x1000000]
#### lspci:
06:01.0 Unassigned class [ff00]: Matrox Electronics Systems Ltd. VIA Framegrabber (rev 01)
#### lspci -n
06:01.0 ff00: 102b:4536 (rev 01)
#### lspci -t
-[0000:00]-+-00.0
           +-01.0-[0a]--+-00.0
           |            \-00.1
           +-03.0-[09]--
           +-05.0-[08]--
           +-07.0-[07]--
           +-09.0-[04-06]--+-00.0-[06]----01.0
#### lspci -v
06:01.0 Unassigned class [ff00]: Matrox Electronics Systems Ltd. VIA Framegrabber (rev 01)
        Control: I/O+ Mem+ BusMaster+ SpecCycle+ MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 64
        Interrupt: pin A routed to IRQ 3
        Region 0: Memory at d8000000 (32-bit, prefetchable) [size=128M]
        Region 1: Memory at fbdff000 (32-bit, non-prefetchable) [size=4K]

Full lspci:
Code: 
00:00.0 Host bridge: Intel Corporation 5520 I/O Hub to ESI Port (rev 13)
00:01.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 1 (rev 13)
00:03.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 3 (rev 13)
00:05.0 PCI bridge: Intel Corporation 5520/X58 I/O Hub PCI Express Root Port 5 (rev 13)
00:07.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 7 (rev 13)
00:09.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 9 (rev 13)
00:13.0 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub I/OxAPIC Interrupt Controller (rev 13)
00:14.0 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub System Management Registers (rev 13)
00:14.1 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub GPIO and Scratch Pad Registers (rev 13)
00:14.2 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub Control Status and RAS Registers (rev 13)
00:14.3 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub Throttle Registers (rev 13)
00:16.0 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13)
00:16.1 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13)
00:16.2 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13)
00:16.3 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13)
00:16.4 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13)
00:16.5 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13)
00:16.6 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13)
00:16.7 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13)
00:1a.0 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4
00:1a.1 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5
00:1a.2 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6
00:1a.7 USB controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2
00:1c.0 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 1
00:1c.4 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 5
00:1d.0 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1
00:1d.1 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2
00:1d.2 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3
00:1d.7 USB controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90)
00:1f.0 ISA bridge: Intel Corporation 82801JIR (ICH10R) LPC Interface Controller
00:1f.2 SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller
00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller
01:01.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] ES1000 (rev 02)
02:00.0 IDE interface: JMicron Technology Corp. JMB368 IDE controller
04:00.0 PCI bridge: Intel Corporation 6700PXH PCI Express-to-PCI Bridge A (rev 09)
04:00.1 PIC: Intel Corporation 6700/6702PXH I/OxAPIC Interrupt Controller A (rev 09)
04:00.2 PCI bridge: Intel Corporation 6700PXH PCI Express-to-PCI Bridge B (rev 09)
04:00.3 PIC: Intel Corporation 6700PXH I/OxAPIC Interrupt Controller B (rev 09)
06:01.0 Unassigned class [ff00]: Matrox Electronics Systems Ltd. VIA Framegrabber (rev 01)
0a:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
0a:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)

I have tried holding the device with stub driver in grub- /etc/default/grub:
Code: 
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on pci-stub.ids=102b:4536"

Startup error.
Code: 
 qm start 100
kvm: -device pci-assign,host=06:01.0,id=hostpci0,bus=pci.0,addr=0x10: Failed to assign device "hostpci0": Operation not permitted
kvm: -device pci-assign,host=06:01.0,id=hostpci0,bus=pci.0,addr=0x10: Device initialization failed.
kvm: -device pci-assign,host=06:01.0,id=hostpci0,bus=pci.0,addr=0x10: Device 'kvm-pci-assign' could not be initialized


I find it strange that no identity map gets created for the device - I assume this is a problem??
 
Last edited:
for posterity.... i will post my own solution ;)
credit goes to this guy:
http://spica-and-roid.blogspot.com/2012/07/howto-kvm-passthrough.html

ran it on 2.6.32-37 kernel, requirements:
/etc/default/grub:
Code: 
[COLOR=#333333]
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on pci-stub.ids=102b:4536"
[/COLOR]
Then run update-grub and reboot.

This would be based on my lspci results from previous post:
Code: 
modprobe pci_stub
echo "102b 4536" > /sys/bus/pci/drivers/pci-stub/new_id 
echo 0000:06:01.0 > /sys/bus/pci/devices/0000\:06\:01.0/driver/unbind
echo 0000:06:01.0 > /sys/bus/pci/drivers/pci-stub/bind


[COLOR=#333333][FONT=monospace]echo 1 > /sys/module/kvm/parameters/allow_unsafe_assigned_interrupts[/FONT][/COLOR]

I had to put the last line in /etc/rc.local to get all this working on a reboot, not sure if someone has a better way.

And the icing on the cake- add the device to your VM.conf: hostpci0: 06:01.0
 
Last edited:
  • Like
Reactions: KB19 and Stoiko Ivanov
You must log in or register to reply here.
Top Bottom