PCI passthrough

S

sk8erbender

New Member
Feb 9, 2016
10
0
1
32
Hello everyone! Ive read wiki and some topics here but I have a problem.
I have a win7 64x virtual machine.
And i have the following pci card in the motherboard MCS9865 Moschip
So far I ve done the following

edit:
#vi /etc/default/grub
change
GRUB_CMDLINE_LINUX_DEFAULT="quiet"
to
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"
then
# update-grub

rebooted system

added
add to /etc/modules

vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

rebooted system

then I did lspci

03.01.00

and I putted it in /etc/pve/qemu-server/<vmid>.cfg

with following
machine: q35
hostpci0: 03:01.0,pcie=1

then I have linux kernel panic system hangs and I have to reboot and remove that from .conf

Need some tips
 
fireon said:
Hello,

https://pve.proxmox.com/wiki/Pci_passthrough#Intel_CPU
after you added the section in grub, you reboot. Have you checked, is iommu on?
Code: 
dmesg | grep -e DMAR -e IOMMU
Click to expand...
Output

[ 0.481516] DMAR: Intel(R) Virtualization Technology for Directed I/O[ 0.481418] DMAR: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff][ 0.481413] DMAR: Prepare 0-16MiB unity mapping for LPC[ 0.481410] DMAR: Setting identity map for device 0000:00:1d.2 [0xdfd08000 - 0xdfd10fff][ 0.481408] DMAR: Setting identity map for device 0000:00:1d.1 [0xdfd08000 - 0xdfd10fff][ 0.481406] DMAR: Setting identity map for device 0000:00:1d.0 [0xdfd08000 - 0xdfd10fff][ 0.481404] DMAR: Setting identity map for device 0000:00:1a.1 [0xdfd02000 - 0xdfd07fff][ 0.481402] DMAR: Setting identity map for device 0000:00:1a.0 [0xdfd02000 - 0xdfd07fff][ 0.481401] DMAR: Setting identity map for device 0000:00:1d.7 [0xdfd32000 - 0xdfd34fff][ 0.481393] DMAR: Setting identity map for device 0000:00:1d.2 [0xdfd32000 - 0xdfd34fff][ 0.481381] DMAR: Setting identity map for device 0000:00:1d.1 [0xdfd32000 - 0xdfd34fff][ 0.481368] DMAR: Setting identity map for device 0000:00:1d.0 [0xdfd32000 - 0xdfd34fff][ 0.481361] DMAR: Setting identity map for device 0000:00:1a.7 [0xdfd32000 - 0xdfd34fff][ 0.481353] DMAR: Setting identity map for device 0000:00:1a.1 [0xdfd32000 - 0xdfd34fff][ 0.481341] DMAR: Setting identity map for device 0000:00:1a.0 [0xdfd32000 - 0xdfd34fff][ 0.481329] DMAR: Setting identity map for device 0000:00:1d.7 [0xdfcff000 - 0xdfd01fff][ 0.481310] DMAR: Setting identity map for device 0000:00:1a.7 [0xdfcfc000 - 0xdfcfefff][ 0.481302] DMAR: Setting RMRR:[ 0.481292] DMAR: dmar0: Using Register based invalidation[ 0.480988] DMAR: No ATSR found[ 0.032088] DMAR: RMRR base: 0x000000dfcfc000 end: 0x000000dfcfefff[ 0.032086] DMAR: RMRR base: 0x000000dfcff000 end: 0x000000dfd01fff[ 0.032085] DMAR: RMRR base: 0x000000dfd32000 end: 0x000000dfd34fff[ 0.032084] DMAR: RMRR base: 0x000000dfd02000 end: 0x000000dfd07fff[ 0.032082] DMAR: RMRR base: 0x000000dfd08000 end: 0x000000dfd10fff[ 0.032081] DMAR: dmar0: reg_base_addr feb03000 ver 1:0 cap c9008020230272 ecap 1000[ 0.032076] DMAR: DRHD base: 0x000000feb03000 flags: 0x1[ 0.032073] DMAR: Host address width 36[ 0.000000] DMAR: IOMMU enabled[ 0.000000] ACPI: DMAR 0x00000000DFDE5000 000128 (v01 INTEL S3200SHV 00000001 MSFT 00000000)
 
Your MCS9865 is a PCI controller, not PCIe. To my knowledge, that cannot be passed-through using vfio. Maybe it works if you additionally passthrough the corresponding PCIe-PCI bridge, which provides the PCI root port for your MCS9865. Or alternatively, maybe the old qemu pci-assign command still works? Not sure, though, how to configure this in Proxmox.
 
oelsi said:
Your MCS9865 is a PCI controller, not PCIe. To my knowledge, that cannot be passed-through using vfio. Maybe it works if you additionally passthrough the corresponding PCIe-PCI bridge, which provides the PCI root port for your MCS9865. Or alternatively, maybe the old qemu pci-assign command still works? Not sure, though, how to configure this in Proxmox.
Click to expand...

it's possible to use vfio without pcie , no prolem

Code: 
hostpci0: 03:01.0
(no machine option)
 
spirit said:
Again :

do you have correct iommu isolation ?
https://pve.proxmox.com/wiki/Pci_passthrough#Verify_IOMMU_isolation
Click to expand...

root@servervm:~# find /sys/kernel/iommu_groups/ -type l
/sys/kernel/iommu_groups/0/devices/0000:00:00.0
/sys/kernel/iommu_groups/1/devices/0000:00:1a.0
/sys/kernel/iommu_groups/1/devices/0000:00:1a.1
/sys/kernel/iommu_groups/1/devices/0000:00:1a.7
/sys/kernel/iommu_groups/2/devices/0000:00:1c.0
/sys/kernel/iommu_groups/2/devices/0000:00:1c.4
/sys/kernel/iommu_groups/2/devices/0000:02:00.0
/sys/kernel/iommu_groups/3/devices/0000:00:1d.0
/sys/kernel/iommu_groups/3/devices/0000:00:1d.1
/sys/kernel/iommu_groups/3/devices/0000:00:1d.2
/sys/kernel/iommu_groups/3/devices/0000:00:1d.7
/sys/kernel/iommu_groups/4/devices/0000:00:1e.0
/sys/kernel/iommu_groups/4/devices/0000:03:01.0
/sys/kernel/iommu_groups/4/devices/0000:03:02.0
/sys/kernel/iommu_groups/5/devices/0000:00:1f.0
/sys/kernel/iommu_groups/5/devices/0000:00:1f.2
/sys/kernel/iommu_groups/5/devices/0000:00:1f.3
 
spirit said:
Again :

do you have correct iommu isolation ?
https://pve.proxmox.com/wiki/Pci_passthrough#Verify_IOMMU_isolation
Click to expand...

cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Xeon(R) CPU E3110 @ 3.00GHz
stepping : 10
microcode : 0xa07
cpu MHz : 3000.000
cache size : 6144 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 2
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm dtherm tpr_shadow vnmi flexpriority
bugs :
bogomips : 5984.98
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Xeon(R) CPU E3110 @ 3.00GHz
stepping : 10
microcode : 0xa07
cpu MHz : 3000.000
cache size : 6144 KB
physical id : 0
siblings : 2
core id : 1
cpu cores : 2
apicid : 1
initial apicid : 1
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm dtherm tpr_shadow vnmi flexpriority
bugs :
bogomips : 5984.98
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:




100.conf

balloon: 512
bootdisk: virtio0
cores: 1
ide2: none,media=cdrom
memory: 1024
name: Win7
net0: virtio=32:32:36:66:38:35,bridge=vmbr0
numa: 0
ostype: win7
smbios1: uuid=eb9c6370-6d2c-4250-b2d3-c741b29bf207
sockets: 1
virtio0: local:100/vm-100-disk-1.vmdk,cache=writeback,size=60G
hostpci0: 03:01.0


lspci



00:00.0 Host bridge: Intel Corporation 3200/3210 Chipset DRAM Controller
00:1a.0 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #4 (rev 02)
00:1a.1 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #5 (rev 02)
00:1a.7 USB controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #2 (rev 02)
00:1c.0 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express Port 1 (rev 02)
00:1c.4 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express Port 5 (rev 02)
00:1d.0 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 (rev 02)
00:1d.1 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2 (rev 02)
00:1d.2 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #3 (rev 02)
00:1d.7 USB controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #1 (rev 02)
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 92)
00:1f.0 ISA bridge: Intel Corporation 82801IR (ICH9R) LPC Interface Controller (rev 02)
00:1f.2 RAID bus controller: Intel Corporation SATA Controller [RAID mode] (rev 02)
00:1f.3 SMBus: Intel Corporation 82801I (ICH9 Family) SMBus Controller (rev 02)
02:00.0 VGA compatible controller: Matrox Electronics Systems Ltd. MGA G200e [Pilot] ServerEngines (SEP1) (rev 02)
03:01.0 Parallel controller: MosChip Semiconductor Technology Ltd. PCI 9865 Multi-I/O Controller
03:02.0 Ethernet controller: Intel Corporation 82541GI Gigabit Ethernet Controller (rev 05)


dmesg | grep ecap
[ 0.032079] DMAR: dmar0: reg_base_addr feb03000 ver 1:0 cap c9008020230272 ecap 1000
 
lspci -v

03:01.0 Parallel controller: MosChip Semiconductor Technology Ltd. PCI 9865 Multi-I/O Controller (prog-if 03 [IEEE1284])
Subsystem: Device a000:2000
Flags: bus master, medium devsel, latency 32, IRQ 17
I/O ports at 1048
I/O ports at 1040
Memory at e1941000 (32-bit, non-prefetchable) [size=4K]
Memory at e1940000 (32-bit, non-prefetchable) [size=4K]
Capabilities: [48] Power Management version 2
Kernel driver in use: parport_pc

lspci -nn

03:01.0 Parallel controller [0701]: MosChip Semiconductor Technology Ltd. PCI 9865 Multi-I/O Controller [9710:9865]
 
sk8erbender said:
root@servervm:~# find /sys/kernel/iommu_groups/ -type l
/sys/kernel/iommu_groups/0/devices/0000:00:00.0
/sys/kernel/iommu_groups/1/devices/0000:00:1a.0
/sys/kernel/iommu_groups/1/devices/0000:00:1a.1
/sys/kernel/iommu_groups/1/devices/0000:00:1a.7
/sys/kernel/iommu_groups/2/devices/0000:00:1c.0
/sys/kernel/iommu_groups/2/devices/0000:00:1c.4
/sys/kernel/iommu_groups/2/devices/0000:02:00.0
/sys/kernel/iommu_groups/3/devices/0000:00:1d.0
/sys/kernel/iommu_groups/3/devices/0000:00:1d.1
/sys/kernel/iommu_groups/3/devices/0000:00:1d.2
/sys/kernel/iommu_groups/3/devices/0000:00:1d.7
/sys/kernel/iommu_groups/4/devices/0000:00:1e.0
/sys/kernel/iommu_groups/4/devices/0000:03:01.0
/sys/kernel/iommu_groups/4/devices/0000:03:02.0
/sys/kernel/iommu_groups/5/devices/0000:00:1f.0
/sys/kernel/iommu_groups/5/devices/0000:00:1f.2
/sys/kernel/iommu_groups/5/devices/0000:00:1f.3
Click to expand...

It'll not work, because you have in group4,
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 92)



That's why your system hang

you can try to add "pcie_acs_override=downstream" in your grub options, to try to split the iommu groups
 
spirit said:
It'll not work, because you have in group4,
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 92)



That's why your system hang

you can try to add "pcie_acs_override=downstream" in your grub options, to try to split the iommu groups
Click to expand...


dmesg | grep ecap
[ 0.032079] DMAR: dmar0: reg_base_addr feb03000 ver 1:0 cap c9008020230272 ecap 1000

IOMMU interrupt remapping For example, "ecap 1000" indicates there is no interrupt remapping support

Should I also try this one -
echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf ???
 
spirit said:
It'll not work, because you have in group4,
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 92)



That's why your system hang

you can try to add "pcie_acs_override=downstream" in your grub options, to try to split the iommu groups
Click to expand...

Added pcie_acs_override=downstream in grub, rebooted also added echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf
but system hangs
 
sk8erbender said:
Added pcie_acs_override=downstream in grub, rebooted also added echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf
but system hangs
Click to expand...

you need to check your iommu groups after theses change, if it's still grouped, you are stuck, your motherboard/cpu is not able to do pci passthrough.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back