Once again: PCI passthrough problems

M

MarkusH

Renowned Member
Nov 30, 2011
62
11
73
Hi,

i try to passthrough a pci device to my vm:
root@master-srv:~# lspci
00:00.0 Host bridge: Intel Corporation 82Q35 Express DRAM Controller (rev 02)
00:01.0 PCI bridge: Intel Corporation 82Q35 Express PCI Express Root Port (rev 02)
00:02.0 VGA compatible controller: Intel Corporation 82Q35 Express Integrated Graphics Controller (rev 02)
00:03.0 Communication controller: Intel Corporation 82Q35 Express MEI Controller (rev 02)
00:03.2 IDE interface: Intel Corporation 82Q35 Express PT IDER Controller (rev 02)
00:03.3 Serial controller: Intel Corporation 82Q35 Express Serial KT Controller (rev 02)
00:19.0 Ethernet controller: Intel Corporation 82566DM-2 Gigabit Network Connection (rev 02)
00:1a.0 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #4 (rev 02)
00:1a.1 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #5 (rev 02)
00:1a.2 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #6 (rev 02)
00:1a.7 USB controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #2 (rev 02)
00:1c.0 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express Port 1 (rev 02)
00:1c.4 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express Port 5 (rev 02)
00:1d.0 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 (rev 02)
00:1d.1 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2 (rev 02)
00:1d.2 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #3 (rev 02)
00:1d.7 USB controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #1 (rev 02)
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 92)
00:1f.0 ISA bridge: Intel Corporation 82801IO (ICH9DO) LPC Interface Controller (rev 02)
00:1f.2 SATA controller: Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode] (rev 02)
00:1f.3 SMBus: Intel Corporation 82801I (ICH9 Family) SMBus Controller (rev 02)
04:07.0 Network controller: Techsan Electronics Co Ltd B2C2 FlexCopII DVB chip / Technisat SkyStar2 DVB card (rev 02)
Click to expand...

I need the last card: 04:07.0. To prevent the Proxmox Server to use the card with kernel based support i have add the following to /etc/modprobe.d/blacklist.conf:
blacklist s5h1420
blacklist dvb_core
blacklist cx24113
blacklist cx24123
blacklist i2c-core
blacklist b2c2_flexcop
blacklist b2c2_flexcop_pci
Click to expand...

Now i add the hospci command to my vm:
root@master-srv:~# cat /etc/pve/qemu-server/101.conf
ide0: local:101/vm-101-disk-1.raw
ostype: l26
memory: 1024
sockets: 1
net0: e1000=0E:1B:56:A6:C2:89,bridge=vmbr0
ide2: local:iso/yavdr64-0.4.0.iso,media=cdrom
name: srv-vdr
bootdisk: ide0
cores: 1
hostpci0: 04:07.0
Click to expand...

But the vm don't start anymore:
*** EHCI support is under development ***
Failed to assign device "hostpci0" : Operation not permitted
kvm: -device pci-assign,host=04:07.0,id=hostpci0,bus=pci.0,addr=0x10: Device 'pci-assign' could not be initialized
TASK ERROR: start failed: command '/usr/bin/kvm -id 101 -chardev socket,id=monitor,path=/var/run/qemu-server/101.mon,server,nowait -mon chardev=monitor,mode=readline -vnc unix:/var/run/qemu-server/101.vnc,x509,password -pidfile /var/run/qemu-server/101.pid -daemonize -readconfig /usr/share/qemu-server/pve-usb.cfg -device usb-tablet,bus=ehci.0,port=6 -device pci-assign,host=04:07.0,id=hostpci0,bus=pci.0,addr=0x10 -name srv-vdr -smp sockets=1,cores=1 -nodefaults -boot menu=on,order=cdn -vga cirrus -tdf -k de -drive file=/var/lib/vz/template/iso/yavdr64-0.4.0.iso,if=none,id=drive-ide2,media=cdrom,aio=native -device ide-drive,bus=ide.1,unit=0,drive=drive-ide2,id=device-ide2 -drive file=/var/lib/vz/images/101/vm-101-disk-1.raw,if=none,id=drive-ide0,aio=native,boot=on -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0,id=device-ide0 -m 1024 -netdev type=tap,id=net0,ifname=tap101i0,script=/var/lib/qemu-server/pve-bridge -device e1000,romfile=,mac=0E:1B:56:A6:C2:89,netdev=net0,bus=pci.0,addr=0x12 -cpuunits 1000' failed: exit code 1
Click to expand...

Have you any ideas?

dmesg on my vm:
dmesg | grep -e DMAR -e IOMMU
[ 0.000000] Intel-IOMMU: enabled
Click to expand...

dmesg on the proxmox server:
root@master-srv:~# dmesg | grep -e DMAR -e IOMMU
ACPI: DMAR 00000000bf1d797b 000D8 (v01 Intel OEMDMAR 00060000 LOHR 00000001)
Intel-IOMMU: enabled
DMAR: Host address width 36
DMAR: DRHD base: 0x000000fed91000 flags: 0x0
IOMMU fed91000: ver 1:0 cap c0000020230270 ecap 1000
DMAR: DRHD base: 0x000000fed92000 flags: 0x0
IOMMU fed92000: ver 1:0 cap c0000020230270 ecap 1000
DMAR: DRHD base: 0x000000fed93000 flags: 0x1
IOMMU fed93000: ver 1:0 cap c9008020230270 ecap 1000
DMAR: RMRR base: 0x000000bf1db000 end: 0x000000bf1fffff
DMAR: No ATSR found
IOMMU 0xfed92000: using Register based invalidation
IOMMU 0xfed91000: using Register based invalidation
IOMMU 0xfed93000: using Register based invalidation
IOMMU: Setting RMRR:
IOMMU: Setting identity map for device 0000:00:1d.0 [0xbf1db000 - 0xbf200000]
IOMMU: Setting identity map for device 0000:00:1d.1 [0xbf1db000 - 0xbf200000]
IOMMU: Setting identity map for device 0000:00:1d.2 [0xbf1db000 - 0xbf200000]
IOMMU: Setting identity map for device 0000:00:1d.7 [0xbf1db000 - 0xbf200000]
IOMMU: Setting identity map for device 0000:00:1a.0 [0xbf1db000 - 0xbf200000]
IOMMU: Setting identity map for device 0000:00:1a.1 [0xbf1db000 - 0xbf200000]
IOMMU: Setting identity map for device 0000:00:1a.2 [0xbf1db000 - 0xbf200000]
IOMMU: Setting identity map for device 0000:00:1a.7 [0xbf1db000 - 0xbf200000]
IOMMU: Prepare 0-16MiB unity mapping for LPC
IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0x1000000]
DMAR:[DMA Write] Request device [00:02.0] fault addr bf800000
DMAR:[fault reason 01] Present bit in root entry is clear
Click to expand...
 
I have tried to get it working - without any success.

The computer i use is a Fujitsu Esprimo 5925 with a Intel Core2Duo.
Any idea?
 
Last edited:
Thanks, I know this document and read it several times. I think my problem is caused by the used hardware. I will try another...

Markus
 
Did you have a look on your PCI-bus-config?
lspci -t
After it's a PCI-Card, you need to passthrough *all* PCI-Devices behind a bridge. On some of my mainboards often the e.g. Firewire was on the same bus. After deaktivating it in Bios, the PT went well.
Regards,
Daniel
 
lspci -t:
root@srv-virt-host:~# lspci -t
-[0000:00]-+-00.0
+-01.0-[01]--
+-02.0
+-03.0
+-03.2
+-03.3
+-19.0
+-1a.0
+-1a.1
+-1a.2
+-1a.7
+-1c.0-[02]--
+-1c.4-[03]--
+-1d.0
+-1d.1
+-1d.2
+-1d.7
+-1e.0-[04]----07.0
+-1f.0
+-1f.2
\-1f.3

I think there is no other device than 04:07.0, isn't it so?

I have already deactivated all unused devices like parallel port, seriell port, soundcard and so on.
 
Hi Alex,

the solution is to passthrough the corresponding PCI-Device of the motherboard. In my case it was shown with the "lspci -t" command (thanks to chefmagier!):
+-1e.0-[04]----07.0
Click to expand...

I also need to passthrough the device "00:1e.0" - thats all.

Markus
 
I don't know!

In my case i have to add two (!) lines with the hostpci command to the vm conf-file: vi /etc/pve/qemu-server/101.conf

hostpci0: 00:1e.0
hostpci1: 04:07.0

Then it works for me.
 
Sorry Alex,

i can't help in you in that way because i don't use proxmox anymore. I have changed to another solution.

Markus
 
Hi,

root@proxmox:~# lspci

00:00.0 Host bridge: Intel Corporation 2nd Generation Core Processor Family DRAM Controller (rev 09)
00:01.0 PCI bridge: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port (rev 09)
00:02.0 VGA compatible controller: Intel Corporation 2nd Generation Core Processor Family Integrated Graphics Controller (rev 09)
00:16.0 Communication controller: Intel Corporation 6 Series/C200 Series Chipset Family MEI Controller #1 (rev 04)
00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2 (rev 05)
00:1b.0 Audio device: Intel Corporation 6 Series/C200 Series Chipset Family High Definition Audio Controller (rev 05)
00:1c.0 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 1 (rev b5)
00:1c.1 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 2 (rev b5)
00:1c.2 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 3 (rev b5)
00:1c.3 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 4 (rev b5)
00:1c.4 PCI bridge: Intel Corporation 82801 PCI Bridge (rev b5)
00:1c.5 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 6 (rev b5)
00:1c.6 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 7 (rev b5)
00:1d.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1 (rev 05)
00:1f.0 ISA bridge: Intel Corporation Z68 Express Chipset Family LPC Controller (rev 05)
00:1f.2 SATA controller: Intel Corporation 6 Series/C200 Series Chipset Family 6 port SATA AHCI Controller (rev 05)
00:1f.3 SMBus: Intel Corporation 6 Series/C200 Series Chipset Family SMBus Controller (rev 05)
03:00.0 Multimedia video controller: Conexant Systems, Inc. CX23885 PCI Video and Audio Decoder (rev 02)
04:00.0 IDE interface: Marvell Technology Group Ltd. Device 91a3 (rev 11)
05:00.0 USB controller: NEC Corporation uPD720200 USB 3.0 Host Controller (rev 04)
06:00.0 PCI bridge: ASMedia Technology Inc. ASM108x PCIe to PCI Bridge Controller (rev 01)
07:00.0 Network controller: AVM GmbH B1 ISDN (rev 01)
08:00.0 USB controller: NEC Corporation uPD720200 USB 3.0 Host Controller (rev 04)
09:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 06)
Click to expand...


root@proxmox:~# lspci -t

-[0000:00]-+-00.0
+-01.0-[01]--
+-02.0
+-16.0
+-1a.0
+-1b.0
+-1c.0-[02]--
+-1c.1-[03]----00.0
+-1c.2-[04]----00.0
+-1c.3-[05]----00.0
+-1c.4-[06-07]----00.0-[07]----00.0
+-1c.5-[08]----00.0
+-1c.6-[09]----00.0
+-1d.0
+-1f.0
+-1f.2
\-1f.3
Click to expand...



i added "hostpci0: 04:00." to my .conf and start the VM:

Status is ok, but VM is stopped:

PCI region 5 at address 0xfbd10000 has size 0x800, which is not a multiple of 4K. You might experience some performance hit due to that.
TASK OK
Click to expand...


i added "hostpci0: 00:1c.2" to my .conf and start my VM:

Device assignment only supports endpoint assignment, device type 4
kvm: -device pci-assign,host=00:1c.2,id=hostpci0,bus=pci.0,addr=0x10: Device 'pci-assign' could not be initialized
TASK ERROR: start failed: command '/usr/bin/kvm -id 100 -chardev 'socket,id=monitor,path=/var/run/qemu-server/100.mon,server,nowait' -mon 'chardev=monitor,mode=readline' -vnc unix:/var/run/qemu-server/100.vnc,x509,password -pidfile /var/run/qemu-server/100.pid -daemonize -usbdevice tablet -device 'pci-assign,host=00:1c.2,id=hostpci0,bus=pci.0,addr=0x10' -name FreeNAS-0.8 -smp 'sockets=1,cores=1' -nodefaults -boot 'menu=on' -vga cirrus -tdf -k en-us -drive 'file=/var/lib/vz/template/iso/FreeNAS-8.0.3-RELEASE-x64.iso,if=none,id=drive-ide2,media=cdrom,aio=native' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=device-ide2,bootindex=200' -drive 'file=/var/lib/vz/images/100/vm-100-disk-1.qcow2,if=none,id=drive-ide0,aio=native' -device 'ide-hd,bus=ide.0,unit=0,drive=drive-ide0,id=device-ide0,bootindex=100' -m 2048 -netdev 'type=tap,id=net0,ifname=tap100i0,script=/var/lib/qemu-server/pve-bridge' -device 'rtl8139,mac=A2:BB:6C:F3:9D:70,netdev=net0,bus=pci.0,addr=0x12,bootindex=300' -cpuunits 1000' failed: exit code 1
Click to expand...


What is the Problem?


Alex
 
Last edited:
Hi Alex,

take a look at the message "... no iommu found ...".

Did you enable the support for iommu etc. virtualiszation functions in the boot parameter? What is the result of "dmesg | grep -e DMAR -e IOMMU"?

You have to enable this to get pci passthrough working. Look here: pci-passthrough
 
Hi Alex,

you must add both corresponding devices to your .conf:

...
hostpci0: 00:1c.2
hostpci1: 04:00.0
Click to expand...

If this doesn't works - i can't help you furthermore. Then you need the help of a professional user or a developer.

Markus
 
Hi Markus,

i tested it, but the same error:

Device assignment only supports endpoint assignment, device type 4
kvm: -device pci-assign,host=00:1c.2,id=hostpci0,bus=pci.0,addr=0x10: Device 'pci-assign' could not be initialized
...
Click to expand...

For my TV-Card "hostpci0: 03:00.0" is enough - it works. :confused:

Alex
 
update:

if i use:

hostpci1: 04:00.0
Click to expand...

(instead of hostpci0: 04:00.0)

-> Start with no errors !

But when i start the VM, the boot process hang over one minute on the bios ("SeaBios....), then boot process go on. The Controller is there, but is not working correctly! When i install the driver for it, Windows (XP) crash with bluescreen.
In a Linux VM, the Controller is not recognized.

if i use:

hostpci1: 04:00.0
hostpci2: 00:1c.2
Click to expand...

-> No Difference!

Any Ideas?


Alex
 
You must log in or register to reply here.
Top Bottom