[SOLVED] Proxmox VE 4.2 iommu error

[Robert Moraru]

Hi,

I`m new to Proxmox VE so please take me easy with my (probably stupid) questions.

My config: Dell T20 Intel E3 version 2x8Gb ram dual chanel and 1x4Gb ram on a second slot.

The error that I get: kernel panic when I enable intel_iommu=on

pveversion -v

proxmox-ve: 4.2-56 (running kernel: 4.4.6-1-pve)
pve-manager: 4.2-15 (running version: 4.2-15/6669ad2c)
pve-kernel-4.4.6-1-pve: 4.4.6-48
pve-kernel-4.4.13-1-pve: 4.4.13-56
lvm2: 2.02.116-pve2
corosync-pve: 2.3.5-2
libqb0: 1.0-1
pve-cluster: 4.0-42
qemu-server: 4.0-83
pve-firmware: 1.1-8
libpve-common-perl: 4.0-70
libpve-access-control: 4.0-16
libpve-storage-perl: 4.0-55
pve-libspice-server1: 0.12.5-2
vncterm: 1.2-1
pve-qemu-kvm: 2.5-19
pve-container: 1.0-70
pve-firewall: 2.0-29
pve-ha-manager: 1.0-32
ksm-control-daemon: 1.2-1
glusterfs-client: 3.5.2-2+deb8u2
lxc-pve: 1.1.5-7
lxcfs: 2.0.0-pve2
cgmanager: 0.39-pve1
criu: 1.6.0-1
zfsutils: 0.6.5.7-pve10~bpo80

dmesg|grep -e DMAR -e IOMMU

[    0.000000] ACPI: DMAR 0x00000000DBFFE510 000080 (v01 INTEL  HSW      00000001 INTL 00000001)
[    0.030144] DMAR: Host address width 39
[    0.030145] DMAR: DRHD base: 0x000000fed90000 flags: 0x1
[    0.030151] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap d2008c20660462 ecap f010da
[    0.030152] DMAR: RMRR base: 0x000000def7c000 end: 0x000000def89fff
[    0.030154] DMAR-IR: IOAPIC id 8 under DRHD base  0xfed90000 IOMMU 0
[    0.030155] DMAR-IR: HPET id 0 under DRHD base 0xfed90000
[    0.030155] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.030299] DMAR-IR: Enabled IRQ remapping in x2apic mode

lspci

00:00.0 Host bridge: Intel Corporation Xeon E3-1200 v3 Processor DRAM Controller (rev 06)

00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller (rev 06)
00:14.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB xHCI (rev 04)
00:16.0 Communication controller: Intel Corporation 8 Series/C220 Series Chipset Family MEI Controller #1 (rev 04)
00:16.3 Serial controller: Intel Corporation 8 Series/C220 Series Chipset Family KT Controller (rev 04)
00:19.0 Ethernet controller: Intel Corporation Ethernet Connection I217-LM (rev 04)
00:1a.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #2 (rev 04)
00:1b.0 Audio device: Intel Corporation 8 Series/C220 Series Chipset High Definition Audio Controller (rev 04)
00:1c.0 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #1 (rev d4)
00:1c.1 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #2 (rev d4)
00:1c.4 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #5 (rev d4)
00:1d.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #1 (rev 04)
00:1f.0 ISA bridge: Intel Corporation C226 Series Chipset Family Server Advanced SKU LPC Controller (rev 04)
00:1f.2 SATA controller: Intel Corporation 8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode] (rev 04)
00:1f.3 SMBus: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller (rev 04)
01:00.0 VGA compatible controller: NVIDIA Corporation Device 0f03 (rev a1)
01:00.1 Audio device: NVIDIA Corporation GF108 High Definition Audio Controller (rev a1)
03:00.0 PCI bridge: Texas Instruments XIO2001 PCI Express-to-PCI Bridge
05:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9128 PCIe SATA 6 Gb/s RAID controller (rev 20)

Basically, I`m trying to do a PCI passthrough for the NVIDIA card for a Windows 7 VM. Also have the Intel P4600 build in graphics.

Also I`m not a computer newbie so BIOS: VT-D & VT-X are enabled and the system is up to date with jessie.

I have fallowed the steps from PROXMOX wiki and I keep getting stuck on inte_iommu=on. It stops proxmox from booting with keyboard frozen (need to push the power button). As a test I have removed all the steps/modifications that I did (from proxmox wiki) and just tried to activate iommu with same result.
I have fixed it by LIVE CD, editing grub.cfg and removing inte_iommu=on after "quiet".


Is there something that I do wrong?
Or if I understood correctly the problem lies in kernel version 4.4 ?
What did you gays do to make it work?
AUR kernel or is it better to wait for a new improved kernel version?

P.S. Sorry if this thread exists, I`m searching for weeks and I have not answered my questions yet.

 

[fireon]

First you don't need a live CD for changing things in grub. In Grubmenu you can edit mit "e" directly. It is easier for test. Hmm, when you everything have done from here: https://pve.proxmox.com/wiki/Pci_passthrough it should normaly works. Have pcie passthrough also running with an DVB card on actual PVE. Maybe your HW support iommu not right...? Can you check your config again please?

 

[Robert Moraru]

First you don't need a live CD for changing things in grub. In Grubmenu you can edit mit "e" directly. It is easier for test. Hmm, when you everything have done from here: https://pve.proxmox.com/wiki/Pci_passthrough it should normaly works. Have pcie passthrough also running with an DVB card on actual PVE. Maybe your HW support iommu not right...? Can you check your config again please?

I have investigated further, and I think the problem is my PCI sata card (Delock Sata Controller Marvell 88SE9128 chipset).
Because of it I keep getting:

kernel: DRHD: handling fault status reg 2
kernel: DMAR:[DMA Read] Request device [00:1f.3]

and

kernel: DRHD: handling fault status reg 3
kernel: DMAR:[DMA Read] Request device [05:00.0]

which represents

00:1f.3 SMBus: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller (rev 04)
05:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9128 PCIe SATA 6 Gb/s RAID controller (rev 20)

from my lspci

I think the main problem is Marvell PCI sata raid controller which is accessing something (illegally) from my CPU and is messing up VT-D groups.

I was hopping to find an simple mod (no kernel compile) that I can do and make it work.
As a last measurement I will upgrade my raid controller to a DELL Perc H310 when I will have the funds.

 

[Robert Moraru]

Solved. Please close thread.

 

[fireon]

Solved. Please close thread.

Please click on Threadtools --> edit, and set the flag to solved.

 

[Robert Moraru]

Please click on Threadtools --> edit, and set the flag to solved.

THX!