I have two different systems running Proxmox VE 8.2.2, a Dell PowerEdge 13th gen server and a Dell PowerEdge 14th gen server.
One is not able to passthrough the card, the other is able.
Both have a Dell HBA330 and Mellanox ConnectX-3 card that are configured for PCI Passthrough.
This is the relevant error on the Dell 13th gen server:
dmesg | grep -e DMAR -e IOMMU
[ 0.010869] ACPI: DMAR 0x000000007BAFE000 0000A0 (v01 DELL PE_SC3 00000001 DELL 00000001)
[ 0.010913] ACPI: Reserving DMAR table memory at [mem 0x7bafe000-0x7bafe09f]
[ 0.076946] DMAR: IOMMU enabled
[ 0.215487] DMAR: Host address width 46
[ 0.215489] DMAR: DRHD base: 0x000000fbffc000 flags: 0x1
[ 0.215501] DMAR: dmar0: reg_base_addr fbffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[ 0.215504] DMAR: RMRR base: 0x00000069f0e000 end: 0x00000071f15fff
[ 0.215509] DMAR: ATSR flags: 0x0
[ 0.215512] DMAR-IR: IOAPIC id 8 under DRHD base 0xfbffc000 IOMMU 0
[ 0.215514] DMAR-IR: IOAPIC id 9 under DRHD base 0xfbffc000 IOMMU 0
[ 0.215516] DMAR-IR: HPET id 0 under DRHD base 0xfbffc000
[ 0.215517] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[ 0.215788] DMAR-IR: IRQ remapping was enabled on dmar0 but we are not in kdump mode
[ 0.215883] DMAR-IR: Enabled IRQ remapping in x2apic mode
[ 0.696318] DMAR: [Firmware Bug]: RMRR entry for device 02:00.0 is broken - applying workaround
[ 0.696344] DMAR: No SATC found
[ 0.696346] DMAR: dmar0: Using Queued invalidation
[ 0.700717] DMAR: Intel(R) Virtualization Technology for Directed I/O
[ 52.218990] vfio-pci 0000:02:00.0: Firmware has requested this device have a 1:1 IOMMU mapping, rejecting configuring the device without a 1:1 mapping. Contact your platform vendor.
Interesting, that is the first Intel system displaying this error. I initially thought it would be limited to AMD systems like my problematic one. I have 2 Intel servers that pass through just fine with this kernel.
For my AMD system, it’s not the X710 or E810 card, also happens with the on-board Intel NICs.