PCI Passthrough fails on boot with DMAR error (Read access is not set)

[mapetik]

On boot, past grub, I will receive the following error which repeats until it times out and Proxmox fails to load:

Error: DMAR: [DMA Read] Request device [02:00.0] fault addr bd42c000 [fault reason 06] PTE Read access is not set

02:00.0 is the following device:

02:00.0 RAID bus controller: LSI Logic / Symbios Logic MegaRAID SAS 2008 [Falcon] (rev 03)
        Subsystem: Dell PERC H310 Mini Monolithics

So obviously it looks like the RAID controller isn't cooperating with IOMMU, but I can't find much info beyond that. I have tried to set intel_iommu=pt in /etc/default/grub and I can get the system to boot that way, however it seems that IOMMU is simply disabled with that config. Ultimately I am trying to pass through a GPU, but this error is occurring without the GPU even installed. Any help would be appreciated.


Hardware:
Dell PowerEdge R720xd
CPU: x2 Intel Xeon E5-2643 v1
Mobo: Dell 0C4Y3R

Software:
BIOS: 2.7.0
OS: Proxmox 6.1
Kernel: 5.0.15-1-pve (also tested w/ the newest 5.4.24-1)
Virtualization: Enabled in UEFI

/etc/default/grub contents:

# If you change this file, run 'update-grub' afterwards to update
# /boot/grub/grub.cfg.
# For full documentation of the options in this file, see:
#   info -f grub -n 'Simple configuration'

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="Proxmox Virtual Environment"
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"
GRUB_CMDLINE_LINUX=""

# Disable os-prober, it might add menu entries for each guest
GRUB_DISABLE_OS_PROBER=true

# Uncomment to enable BadRAM filtering, modify to suit your needs
# This works with Linux (no patch required) and with any kernel that obtains
# the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...)
#GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef"

# Uncomment to disable graphical terminal (grub-pc only)
#GRUB_TERMINAL=console

# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command `vbeinfo'
#GRUB_GFXMODE=640x480

# Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux
#GRUB_DISABLE_LINUX_UUID=true

# Disable generation of recovery mode menu entries
GRUB_DISABLE_RECOVERY="true"

# Uncomment to get a beep at grub start
#GRUB_INIT_TUNE="480 440 1"

/etc/modules contents:

vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

 

[wolfgang]

Hi,

have you enabled IOMMU in the BIOS?
I don't know how Dell is masking this feature in the bios.

 

[rynbrn]

I'm having this issue as well. Also on R720xd.

iommu is enabled, here's the doc on that https://www.dell.com/support/kbdoc/...g-sr-iov-technology-on-dell-poweredge-servers

 

[rynbrn]

curious if op ever worked it out

 

[rynbrn]

possibly related https://forum.proxmox.com/threads/enabling-iommu-on-pve6-zfs-compared-to-pve5-4-ext4.56111/

 

[WorldDrknss]

I think it's just an issue with the LSI Logic / Symbios Logic MegaRAID SAS 2008 [Falcon] (rev 03) cards. I have tried everything to get this work.

I think it's related to the fact that this card can work in both RAID and NON-RAID (Passthrough) modes. When IOMMU is enabled it doesn't see your raid configurations and produces this error.