[SOLVED] PCI Passthrough issues

T

totalimpact

Renowned Member
Dec 12, 2010
133
19
83
I am having some trouble, I am doing this in a lab environment, and previously had it working 100% I took some scratch notes on the setup, and in an emergency my lab system had to be re-purposed for something else (formatted), some months later I am trying to get back to my project and it will not work at all - using the same exact server I started on.

I am trying to get a PCI (not express) card working, it is a Matrox Meteor-II Framegrabber, kind of like a serial video capture card - not a video card.

Now running latest 3.4 PVE, tried both 2.6.32-37 and 3.10.0-8 kernels.

I have tried changing PCI slots. Now I am at a loss. I have seen some posts on needing to pass the entire pci bridge, but having trouble isolating it from other devices on the bridge.

Code: 
#### dmesg | grep -e DMAR -e IOMMU
ACPI: DMAR 00000000bf77e0e0 00120 (v01    AMI  OEMDMAR 00000001 MSFT 00000097)
Intel-IOMMU: enabled
dmar: IOMMU 0: reg_base_addr fbffe000 ver 1:0 cap c90780106f0462 ecap f020f6
IOMMU 0xfbffe000: using Queued invalidation
IOMMU: Setting RMRR:
IOMMU: Setting identity map for device 0000:00:1d.0 [0xbf7ec000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1d.1 [0xbf7ec000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1d.2 [0xbf7ec000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1d.7 [0xbf7ec000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1a.0 [0xbf7ec000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1a.1 [0xbf7ec000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1a.2 [0xbf7ec000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1a.7 [0xbf7ec000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1d.0 [0xe6000 - 0xea000]
IOMMU: Setting identity map for device 0000:00:1d.1 [0xe6000 - 0xea000]
IOMMU: Setting identity map for device 0000:00:1d.2 [0xe6000 - 0xea000]
IOMMU: Setting identity map for device 0000:00:1d.7 [0xe6000 - 0xea000]
IOMMU: Setting identity map for device 0000:00:1a.0 [0xe6000 - 0xea000]
IOMMU: Setting identity map for device 0000:00:1a.1 [0xe6000 - 0xea000]
IOMMU: Setting identity map for device 0000:00:1a.2 [0xe6000 - 0xea000]
IOMMU: Setting identity map for device 0000:00:1a.7 [0xe6000 - 0xea000]
IOMMU: Prepare 0-16MiB unity mapping for LPC
IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0x1000000]
#### lspci:
06:01.0 Unassigned class [ff00]: Matrox Electronics Systems Ltd. VIA Framegrabber (rev 01)
#### lspci -n
06:01.0 ff00: 102b:4536 (rev 01)
#### lspci -t
-[0000:00]-+-00.0
           +-01.0-[0a]--+-00.0
           |            \-00.1
           +-03.0-[09]--
           +-05.0-[08]--
           +-07.0-[07]--
           +-09.0-[04-06]--+-00.0-[06]----01.0
#### lspci -v
06:01.0 Unassigned class [ff00]: Matrox Electronics Systems Ltd. VIA Framegrabber (rev 01)
        Control: I/O+ Mem+ BusMaster+ SpecCycle+ MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 64
        Interrupt: pin A routed to IRQ 3
        Region 0: Memory at d8000000 (32-bit, prefetchable) [size=128M]
        Region 1: Memory at fbdff000 (32-bit, non-prefetchable) [size=4K]

Full lspci:
Code: 
00:00.0 Host bridge: Intel Corporation 5520 I/O Hub to ESI Port (rev 13)
00:01.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 1 (rev 13)
00:03.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 3 (rev 13)
00:05.0 PCI bridge: Intel Corporation 5520/X58 I/O Hub PCI Express Root Port 5 (rev 13)
00:07.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 7 (rev 13)
00:09.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 9 (rev 13)
00:13.0 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub I/OxAPIC Interrupt Controller (rev 13)
00:14.0 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub System Management Registers (rev 13)
00:14.1 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub GPIO and Scratch Pad Registers (rev 13)
00:14.2 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub Control Status and RAS Registers (rev 13)
00:14.3 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub Throttle Registers (rev 13)
00:16.0 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13)
00:16.1 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13)
00:16.2 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13)
00:16.3 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13)
00:16.4 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13)
00:16.5 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13)
00:16.6 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13)
00:16.7 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13)
00:1a.0 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4
00:1a.1 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5
00:1a.2 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6
00:1a.7 USB controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2
00:1c.0 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 1
00:1c.4 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 5
00:1d.0 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1
00:1d.1 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2
00:1d.2 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3
00:1d.7 USB controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90)
00:1f.0 ISA bridge: Intel Corporation 82801JIR (ICH10R) LPC Interface Controller
00:1f.2 SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller
00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller
01:01.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] ES1000 (rev 02)
02:00.0 IDE interface: JMicron Technology Corp. JMB368 IDE controller
04:00.0 PCI bridge: Intel Corporation 6700PXH PCI Express-to-PCI Bridge A (rev 09)
04:00.1 PIC: Intel Corporation 6700/6702PXH I/OxAPIC Interrupt Controller A (rev 09)
04:00.2 PCI bridge: Intel Corporation 6700PXH PCI Express-to-PCI Bridge B (rev 09)
04:00.3 PIC: Intel Corporation 6700PXH I/OxAPIC Interrupt Controller B (rev 09)
06:01.0 Unassigned class [ff00]: Matrox Electronics Systems Ltd. VIA Framegrabber (rev 01)
0a:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
0a:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)

I have tried holding the device with stub driver in grub- /etc/default/grub:
Code: 
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on pci-stub.ids=102b:4536"

Startup error.
Code: 
 qm start 100
kvm: -device pci-assign,host=06:01.0,id=hostpci0,bus=pci.0,addr=0x10: Failed to assign device "hostpci0": Operation not permitted
kvm: -device pci-assign,host=06:01.0,id=hostpci0,bus=pci.0,addr=0x10: Device initialization failed.
kvm: -device pci-assign,host=06:01.0,id=hostpci0,bus=pci.0,addr=0x10: Device 'kvm-pci-assign' could not be initialized


I find it strange that no identity map gets created for the device - I assume this is a problem??
 
Last edited:
for posterity.... i will post my own solution ;)
credit goes to this guy:
http://spica-and-roid.blogspot.com/2012/07/howto-kvm-passthrough.html

ran it on 2.6.32-37 kernel, requirements:
/etc/default/grub:
Code: 
[COLOR=#333333]
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on pci-stub.ids=102b:4536"
[/COLOR]
Then run update-grub and reboot.

This would be based on my lspci results from previous post:
Code: 
modprobe pci_stub
echo "102b 4536" > /sys/bus/pci/drivers/pci-stub/new_id 
echo 0000:06:01.0 > /sys/bus/pci/devices/0000\:06\:01.0/driver/unbind
echo 0000:06:01.0 > /sys/bus/pci/drivers/pci-stub/bind


[COLOR=#333333][FONT=monospace]echo 1 > /sys/module/kvm/parameters/allow_unsafe_assigned_interrupts[/FONT][/COLOR]

I had to put the last line in /etc/rc.local to get all this working on a reboot, not sure if someone has a better way.

And the icing on the cake- add the device to your VM.conf: hostpci0: 06:01.0
 
Last edited:
  • Like
Reactions: KB19 and Stoiko Ivanov
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back