Interrupt Mapping Being Disabled

[twinrotor]

Moved my hardware from a working system with a HBA passed though to a new system. Fresh install, I'm getting error listed below.

Old system:
Dell T5500 x5675, H710 IT mode

New System:
HP DL350 G6, same x5675s, same H710

Both are Intel 5500 chip sets. Worked fine on the Dell, below is what I get with the HP

root@pve:~# dmesg | grep -e DMAR -e IOMMU
[    0.007967] ACPI: DMAR 0x00000000E761FE80 000172 (v01 HP     ProLiant 00000001 \xd2?   0000162E)
[    0.008010] ACPI: Reserving DMAR table memory at [mem 0xe761fe80-0xe761fff1]
[    0.538794] DMAR: IOMMU enabled
[    1.217002] DMAR-IR: This system BIOS has enabled interrupt remapping
[    1.584689] DMAR: Host address width 39
[    1.584691] DMAR: DRHD base: 0x000000efffe000 flags: 0x1
[    1.584702] DMAR: dmar0: reg_base_addr efffe000 ver 1:0 cap c90780106f0462
ecap f0207e
[    1.584705] DMAR: RMRR base: 0x000000e77fc000 end: 0x000000e77fdfff
[    1.584707] DMAR: RMRR base: 0x000000e77f5000 end: 0x000000e77fafff
[    1.584709] DMAR: RMRR base: 0x000000e763e000 end: 0x000000e763ffff
[    1.584710] DMAR: ATSR flags: 0x0
[    1.584751] DMAR: No SATC found
[    1.584757] DMAR: dmar0: Using Queued invalidation
[    1.603403] DMAR: Intel(R) Virtualization Technology for Directed I/O
[    8.274770] DMAR: DRHD: handling fault status reg 2
[    8.274786] DMAR: [DMA Read NO_PASID] Request device [00:1e.0] fault addr 0x1000 [fault reason 0x06] PTE Read access is not set
[    9.150173] AMD-Vi: AMD IOMMUv2 functionality not available on this system
- This is not a bug.

root@pve:~# dmesg | grep 'remapping'
[    1.217002] DMAR-IR: This system BIOS has enabled interrupt remapping
               interrupt remapping is being disabled.
Please
root@pve:~#

I tried the mapping bypass string and no change. Is there something in the HP BIOS I'm missing?

EDIT: This is the error I get trying to pass through a HBA

kvm: -device vfio-pci,host=0000:0b:00.0,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0: vfio 0000:0b:00.0: failed to setup container for group 30: Failed to set iommu for container: Operation not permitted
TASK ERROR: start failed: QEMU exited with code 1

Thanks,
Ben

 

[twinrotor]

I think I found my problem. Booooooo

[  924.798302] vfio-pci 0000:0b:00.0: DMAR: Device is ineligible for IOMMU domain attach due to platform RMRR requirement.  Contact your platform vendor.
[ 1460.214848] vfio-pci 0000:0b:00.0: DMAR: Device is ineligible for IOMMU domain attach due to platform RMRR requirement.  Contact your platform vendor.

 

[Neobin]

Since Proxmox kernel 6.2.16-13: [1], a patch is included to "add override parameter for RMRR relaxation": [2].

Maybe this helps?

For reference:

• https://bugzilla.proxmox.com/show_bug.cgi?id=4707
• https://github.com/kiler129/relax-intel-rmrr

[1] https://git.proxmox.com/?p=pve-kern...;hpb=8ff596f2d384c76fd3fcbaa4ff44ca3bb1af831a
[2] https://git.proxmox.com/?p=pve-kernel.git;a=commit;h=1acfcad2f362f9ff0b53dc23ba0933bade18aa12

 

[twinrotor]

Thanks!

The first thing Google took me to was the killer129 github. I did not find the bug report though! Thanks so much! I really was very disheartened by this yesterday, after taking the time to swap all my parts from the Dell to the HP. I'll find out this afternoon if the patch works on my hardware. Thanks again!

 

[twinrotor]

Added relax_rmrr to /etc/default/grub

Added options vfio_iommu_type1 allow_unsafe_interrupts=1 to /etc/modprobe.d/iommu_unsafe_interrupts.conf

Works perfectly!