ProxMox VE 1.5 2.6.32-2-7 (10jul07) doesn't support PCI Pass-Through

ryansinn

New Member
Jul 12, 2010
8
0
1
I've tried for about 12 hours+ to get ProxMox / qemu-server to support PCI Pass Through to a KVM running a Windows 2000 Guest, but no luck.

I'm running a CPU0: Intel(R) Xeon(R) CPU E5506 @ 2.13GHz stepping 05

The PCI Hardware is a MOXA PCI Serial Card. It was detected in lspci and lsmod showed that the driver was loaded (mxser) -- so I blacklisted the driver in /etc/modprobe.d/blacklist and restarted.

Yet when attempting to start the KVM with the following code in the /etc/qemu-server/101.cfg file:

Code:
args: -pcidevice host="29:01.0"
The system errors out with:

Code:
No IOMMU found. Unable to assign device "29:01.0"
Failed to  deassign device "29:01.0" : Invalid argument
Error initializing device  pci-assign
start failed: command '/usr/bin/kvm -monitor  unix:/var/run/qemu-server/101.mon,server,nowait -vnc  unix:/var/run/qemu-server/101.vnc,password -pidfile /var/run/qemu-server/101.pid  -daemonize -usbdevice tablet -name rgv -smp sockets=1,cores=1 -nodefaults -boot  menu=on,order=dc -vga cirrus -tdf -localtime -rtc-td-hack -k en-us -drive  file=/var/lib/vz/images/101/vm-101-disk-1.raw,if=ide,index=0,boot=on -drive  if=ide,index=2,media=cdrom -m 1024 -net  tap,vlan=0,ifname=vmtab101i0,script=/var/lib/qemu-server/bridge-vlan -net  nic,vlan=0,model=rtl8139,macaddr=EA:B7:9C:51:84:4D -pcidevice host=29:01.0'  failed with exit code 1
if I do pcihost: 29:01.0 the guest just refuses to start w/o any error...

if I grep 'dmesg' for info I see DMAR, but no IOMMU line.
ACPI: DMAR 000000007f7f0000 000D8 (v01 IBM THURLEY 00000001 IBM 01000013)


I went onto ProxMox's site and installed the debs from pvetest/ that are dated July 07, 2010.


Is the kernel not compiled to support Intel Vt-D or what am I missing?

According to what I can find on Intel's site... the lack of a dmesg line stating:
Code:
IOMMU feb03000: ver 1:0 cap c9008020e30260 ecap 1000
Means that IOMMU isn't built into the kernel...

I also checked /proc/cpuinfo
the "vmd" flag is listed on the processors / cores

Thoughts on this?


Here's the walk thru from linux-kvm ... clearly IOMMU isn't working...
http://www.linux-kvm.org/page/How_to_assign_devices_with_VT-d_in_KVM
 
According to what I can find on Intel's site... the lack of a dmesg line stating:
Code:
IOMMU feb03000: ver 1:0 cap c9008020e30260 ecap 1000
Means that IOMMU isn't built into the kernel...

Really? - what kernel config option is that?
 
Dietmar --

I linked to the linux-kvm article explaining the proper method for compiling a kernel for IOMMU support.


  • set "Bus options (PCI etc.)" -> "Support for DMA Remapping Devices" to "*"
  • set "Bus options (PCI etc.)" -> "Enable DMA Remapping Devices" to "*"
  • set "Bus options (PCI etc.)" -> "PCI Stub driver" to "*"
  • optional setting:
set "Bus options (PCI etc.)" -> "Support for Interrupt Remapping" to "*"


Are these options set in the 2.6.32-2 kernel compile?
 
I'm glad you can read. I have personally volunteered and funded many organizations that are working towards eliminating global illiteracy and am glad to see our efforts bore fruit.

What are the results when you run a:

Code:
dmesg | grep -e DMAR -e IOMMU

I've got 3 different Vt-D supported pieces of hardware an none of them register an IOMMU line in DMESG.
 
wait - although I can read, I forgot to upload the kernel with that fix - sorry. Just give me a few minutes.
 
Thanks -- let me know where / when you upload it and I'll test it and let you know if our pci pass through attempts and dmesg results improve.
 

Code:
proxmox01:~# dmesg | grep -e DMAR -e IOMMU
ACPI: DMAR 000000007f7f0000 000D8 (v01 IBM    THURLEY  00000001 IBM  01000013)
DMAR: Host address width 40
DMAR: DRHD base: 0x000000fe710000 flags: 0x1
IOMMU fe710000: ver 1:0 cap c90780106f0462 ecap f020fe
DMAR: RMRR base: 0x0000007d8b0000 end: 0x0000007d92ffff
DMAR: ATSR flags: 0x0
IOMMU 0xfe710000: using Queued invalidation
IOMMU: Setting RMRR:
IOMMU: Setting identity map for device 0000:00:1a.0 [0x7d8b0000 - 0x7d930000]
IOMMU: Setting identity map for device 0000:00:1a.1 [0x7d8b0000 - 0x7d930000]
IOMMU: Setting identity map for device 0000:00:1a.7 [0x7d8b0000 - 0x7d930000]
IOMMU: Setting identity map for device 0000:00:1d.0 [0x7d8b0000 - 0x7d930000]
IOMMU: Setting identity map for device 0000:00:1d.1 [0x7d8b0000 - 0x7d930000]
IOMMU: Setting identity map for device 0000:00:1d.2 [0x7d8b0000 - 0x7d930000]
IOMMU: Setting identity map for device 0000:00:1d.7 [0x7d8b0000 - 0x7d930000]
IOMMU: Prepare 0-16MiB unity mapping for LPC
IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0x1000000]

Look at that. Rebooted with the new kernel package installed and it looks like -8 kernel adds IOMMU support correctly.

I will test it with our PCI device later today / tonight.
 
Forum software must have thrown away all the "Thanks, proxmox staff" in this thread...
i think it's a bug, perhaps? Dietmar have you checked the related kernel option? ;)

Marco
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!