Issues with iommu with 3.10 kernel in pvetest repo

[mcflym]

root@proxmox:/sys/bus/pci/devices/0000:01:00.0/iommu_group/devices# cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-3.10.0-4-pve root=/dev/mapper/pve-root ro quiet pcie_acs_override=downstream intel_iommu=on

and still in the same group????

 

[XueSheng]

Did you apply the pcie_acs_override patch to the kernel? The commandline itself is useless if the kernel doesn't provide this function. If you don't want to build the kernel yourself, I can provide you the latest git kernel with patch applied.

 

[mcflym]

argh... of course i did NOT

if you can provide the latest kernel with the patch please feel free to share it with me :-D

THANK YOU!

 

[XueSheng]

I didn't figure out how to send private messages... so here's the link* to latest git kernel (3.10) with pcie_acs_override patch applied. Patch can be found here. Download is valid until tomorrow. Password is: proxmox

* Update:
Link was removed because it's not valid anymore.

 

[mcflym]

thank you so much!

Works perfectly!

Each device has his own group now.

 

[maxprox]

Latest testing kernel 3.10 works great for me.

1. I've enabled allow_unsafe_int
.

and so on.....

Hello XueSheng

Thank you,
and yes, it works for me too (very happy)

the path to the modprobe-file is

# cat /etc/modprobe.d/iommu_unsafe_interrupts.conf
options vfio_iommu_type1 allow_unsafe_interrupts=1

with ".d"


I use also machine model q35 without pcie flag.

machine: q35
hostpci0: 09:00.0,driver=vfio

The remaining settings like mentioned in the wiki:

https://pve.proxmox.com/wiki/Pci_passthrough

Many thanks!
maxprox

 

[spirit]

Thanks for the reports, I have updated the wiki for

options vfio_iommu_type1 allow_unsafe_interrupts=1

 

[spirit]

Hi,
pcie_acs_override patch has been added to pve-kernel 3.10 in pvetest repository.

Please test

 

[XueSheng]

the path to the modprobe-file is

# cat /etc/modprobe.d/iommu_unsafe_interrupts.conf
options vfio_iommu_type1 allow_unsafe_interrupts=1

with ".d"

Thanks for this hint! That happens if you don't copy&paste commandlines .

Thanks for the reports, I have updated the wiki for

options vfio_iommu_type1 allow_unsafe_interrupts=1

Just saw that the wiki also contains the wrong path to "modprobe.d".
@Spirit: Would you mind correcting this?

As a general note to "pcie_acs_override". I was wondering why I needed this patch with the earlier 3.10 kernels, but now I don't need it anymore. This is because the rhel kernel was patched with quirks for some devices (see LKML as reference).

 

[spirit]

@Spirit: Would you mind correcting this?

Done !

 

[maxprox]

Thanks for the reports, I have updated the wiki for

options vfio_iommu_type1 allow_unsafe_interrupts=1

Hello,

today, after the last Updates:

Will install 2 packages, and remove 0 packages.
===============================================================================
[UPGRADE] pve-kernel-3.10.0-4-pve:amd64 3.10.0-14 -> 3.10.0-15
[UPGRADE] pve-manager:amd64 3.2-22 -> 3.2-23
===============================================================================

My PCIe TV work only with this two lines in the vmid.conf

hostpci0: 02:00.0,pcie=1,driver=vfio
machine: q35

This grub boot line in /etc/default/grub

...
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"
...

And WITHOUT this file:
/etc/modprobe.d/iommu_unsafe_interrupts.conf
with this line

options vfio_iommu_type1 allow_unsafe_interrupts=1

With this file and its option I get the following error:

kvm: -device vfio-pci,host=02:00.0,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0: vfio: Error: Failed to setup INTx fd: Device or resource busy
kvm: -device vfio-pci,host=02:00.0,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0: Device initialization failed.
kvm: -device vfio-pci,host=02:00.0,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0: Device 'vfio-pci' could not be initialized
start failed:
...

Without the iommu and interrupt option it works fine!

Thank you and regards
maxprox

 

[kobuki]

I'm running into similar problems with my Intel DQ67SW-based home server runing PVE. The problems started with upgrading from 3.2 to 3.3, kernel went from 2.6.32-30-pve to 2.6.32-32-pve. I'll try to sum up what happened. The card I'm passing through is an IBM Serveraid M1015 in IT mode for ZFS in a VM. Relevant config line is simply: "hostpci0: 01:00.0", nothing else.

1. PVE 3.2, kernel 2.6.32-30-pve, pve-qemu-kvm 1.7-8, kernel parameter intel_iommu=on -- working

2. PVE 3.3, kernel 2.6.32-32-pve, pve-qemu-kvm 2.1-8, kernel parameter intel_iommu=on -- NOT working, similar problem as here: http://forum.proxmox.com/threads/18366-Kernel-2-6-32-28-not-booting-with-iommu-and-pci-passthrough, message endlessly repeating on the console, then server freezes with a kernel panic, see image

3. same as above, but kernel parameter intel_iommu=igfx_off -- NOT working, system can boot and is stable, but same problem as first post in this thread, no IOMMU found

4. same as above, but kernel parameter intel_iommu=on, and pve-qemu-kvm downgraded to 1.7-8 -- working again

Since I need OVZ, I can't upgrade to 3.10 on this server, so I can't use vfio+pcie here. I'd like to use and test the new features in KVM 2.1 though. Any ideas are very welcome.

 

[eric_f]

Hi,
pcie_acs_override patch has been added to pve-kernel 3.10 in pvetest repository.

Please test

Is this in 3.10.0-5? I am using that kernel and still having issues with devices not having their own group.