Interrupt Mapping Disabled

[twinrotor]

Moved my hardware from a working system with a HBA passed though to a new system. Fresh install, I'm getting error listed below.

Old system:
Dell T5500 x5675, H710 IT mode

New System:
HP DL350 G6, same x5675s, same H710

Both are Intel 5500 chip sets. Worked fine on the Dell, below is what I get with the HP

root@pve:~# dmesg | grep -e DMAR -e IOMMU
[    0.007967] ACPI: DMAR 0x00000000E761FE80 000172 (v01 HP     ProLiant 00000001 \xd2?   0000162E)
[    0.008010] ACPI: Reserving DMAR table memory at [mem 0xe761fe80-0xe761fff1]
[    0.538794] DMAR: IOMMU enabled
[    1.217002] DMAR-IR: This system BIOS has enabled interrupt remapping
[    1.584689] DMAR: Host address width 39
[    1.584691] DMAR: DRHD base: 0x000000efffe000 flags: 0x1
[    1.584702] DMAR: dmar0: reg_base_addr efffe000 ver 1:0 cap c90780106f0462
ecap f0207e
[    1.584705] DMAR: RMRR base: 0x000000e77fc000 end: 0x000000e77fdfff
[    1.584707] DMAR: RMRR base: 0x000000e77f5000 end: 0x000000e77fafff
[    1.584709] DMAR: RMRR base: 0x000000e763e000 end: 0x000000e763ffff
[    1.584710] DMAR: ATSR flags: 0x0
[    1.584751] DMAR: No SATC found
[    1.584757] DMAR: dmar0: Using Queued invalidation
[    1.603403] DMAR: Intel(R) Virtualization Technology for Directed I/O
[    8.274770] DMAR: DRHD: handling fault status reg 2
[    8.274786] DMAR: [DMA Read NO_PASID] Request device [00:1e.0] fault addr 0x1000 [fault reason 0x06] PTE Read access is not set
[    9.150173] AMD-Vi: AMD IOMMUv2 functionality not available on this system
- This is not a bug.

root@pve:~# dmesg | grep 'remapping'
[    1.217002] DMAR-IR: This system BIOS has enabled interrupt remapping
               interrupt remapping is being disabled.
Please
root@pve:~#

I tried the mapping bypass string and no change. Is there something in the HP BIOS I'm missing?

EDIT: This is the error I get trying to pass through a HBA

kvm: -device vfio-pci,host=0000:0b:00.0,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0: vfio 0000:0b:00.0: failed to setup container for group 30: Failed to set iommu for container: Operation not permitted
TASK ERROR: start failed: QEMU exited with code 1

Thanks,
Ben

 

[no-usernames-left]

Try echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf and reboot.

Source: https://pve.proxmox.com/wiki/PCI_Passthrough#Verifying_IOMMU_parameters

 

[leesteken]

Please don't double post the same question: https://forum.proxmox.com/threads/interrupt-mapping-being-disabled.137834/

 

[twinrotor]

Sorry, I didn't know I had double posted! Resolved in other thread. Thanks!

 

[leesteken]

Sorry, I didn't know I had double posted! Resolved in other thread. Thanks!

Really? Please be aware that people spend their time here freely and without any compensation. You can easily find your own posts (unless they're blocked by the spam filter) and do try to prevent other people from wasting their time on threads that already have successful answers.

 

[twinrotor]

Really? Please be aware that people spend their time here freely and without any compensation. You can easily find your own posts (unless they're blocked by the spam filter) and do try to prevent other people from wasting their time on threads that already have successful answers.

Really. I made this post, returned to edit it and it was no where to be found. I assumed I navigated away from the site before it posted. I even went as far as clearing browser cache, to no avail. It won't happen again.

I understand how forums work. It was an honest mistake. I get you're frustration, but I dont understand your tone.

Thank you for your work here and I hope I do not cause you any further grief.

Ben

 

[Alecz]

Try echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf and reboot.

Source: https://pve.proxmox.com/wiki/PCI_Passthrough#Verifying_IOMMU_parameters

Did this work?
I had the same message, added the conf file, but still see:

[    0.333230] DMAR-IR: This system BIOS has enabled interrupt remapping
               on a chipset that contains an erratum making that
               feature unstable.  To maintain system stability
               interrupt remapping is being disabled.  Please
               contact your BIOS vendor for an update

 

[no-usernames-left]

Did this work?
I had the same message, added the conf file, but still see:

[    0.333230] DMAR-IR: This system BIOS has enabled interrupt remapping
               on a chipset that contains an erratum making that
               feature unstable.  To maintain system stability
               interrupt remapping is being disabled.  Please
               contact your BIOS vendor for an update

Did you run the command exactly as shown? If not, what is the name and path of the file you created, and what are the exact contents?

Did you reboot afterwards? Does the file still exist after a reboot?

If all else fails, try the RMRR tip here:
https://forum.proxmox.com/threads/interrupt-mapping-being-disabled.137834/post-614409

 

[Alecz]

I ran the command exactly as shown:

echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf

And rebooted, and verified the file exists and that it shows active at:

 cat /sys/module/vfio_iommu_type1/parameters/allow_unsafe_interrupts
 Y

So I am pretty confident it is active, but it seems the kernel still refused to activate it.

Will give that a try.

 

[Alecz]

Update:
I did the last resort workaround:
My Groub line is now like this:

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on,relax_rmrr iommu=pt vfio_iommu_type1.allow_unsafe_interrupts=1"

I ran `update-grub` and after reboot I have this:

root@pve:~# dmesg | grep 'remapping'
[    0.333543] DMAR-IR: This system BIOS has enabled interrupt remapping
               interrupt remapping is being disabled.  Please

However:

root@pve:~# dmesg | grep -e DMAR -e IOMMU -e AMD-Vi
[    0.024863] ACPI: DMAR 0x00000000CF3B3668 0001E0 (v01 DELL   PE_SC3   00000001 DELL 00000001)
[    0.024974] ACPI: Reserving DMAR table memory at [mem 0xcf3b3668-0xcf3b3847]
[    0.107162] DMAR: IOMMU enabled
[    0.107164] DMAR: Intel-IOMMU: assuming all RMRRs are relaxable. This can lead to instability or data loss
...
[    0.711110] DMAR: Intel(R) Virtualization Technology for Directed I/O

So IOMMU is enabled despite remapping being disabled.

Furthermore, checking the IOMMU groups gives distinct groups:

root@pve:~# pvesh get /nodes/pve/hardware/pci --pci-class-blacklist ""
┌──────────┬────────┬──────────────┬────────────┬────────┬─────────────────────────────────────────────────────────────────────
│ class    │ device │ id           │ iommugroup │ vendor │ device_name                                                         
╞══════════╪════════╪══════════════╪════════════╪════════╪═════════════════════════════════════════════════════════════════════
│ 0x01018f │ 0x2921 │ 0000:00:1f.2 │         14 │ 0x8086 │ 82801IB (ICH9) 2 port SATA Controller [IDE mode]                   
├──────────┼────────┼──────────────┼────────────┼────────┼─────────────────────────────────────────────────────────────────────
│ 0x010700 │ 0x0072 │ 0000:05:00.0 │         16 │ 0x1000 │ SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon]                       
├──────────┼────────┼──────────────┼────────────┼────────┼─────────────────────────────────────────────────────────────────────
│ 0x020000 │ 0x1639 │ 0000:01:00.0 │         15 │ 0x14e4 │ NetXtreme II BCM5709 Gigabit Ethernet                               
├──────────┼────────┼──────────────┼────────────┼────────┼─────────────────────────────────────────────────────────────────────
│ 0x020000 │ 0x1639 │ 0000:01:00.1 │         15 │ 0x14e4 │ NetXtreme II BCM5709 Gigabit Ethernet                               
├──────────┼────────┼──────────────┼────────────┼────────┼─────────────────────────────────────────────────────────────────────
│ 0x030000 │ 0x0532 │ 0000:08:03.0 │         13 │ 0x102b │ MGA G200eW WPCM450                                                 
...

So for example, the LSI SAS2008 is in its own group.

Finally, in the UI, now I can add PCI device to VM.

Time to see how this works....

 

[no-usernames-left]

Finally, in the UI, now I can add PCI device to VM.

Time to see how this works....

Let us know!