Has anyone seen DMA PTE / MegaRAID controller resets after upgrading to recent Proxmox 8 kernels?

[tycjan]

We recently upgraded several 2-node Proxmox clusters from kernel 6.8.12-20-pve to 6.8.12-29-pve and started seeing instability on one node in multiple clusters.

Environment:

• Proxmox VE 8.x
• Dell PowerEdge R350
• Dell PERC H345 RAID controller (megaraid_sas)
• 2-node clusters with Corosync + QDevice + GlusterFS
• Intel VT-d enabled

Observed symptoms:

• Corosync node drops and rejoins
• Unexpected node reboots in some locations
• Filesystem recovery required after reboot on a few hosts
• MegaRAID controller resets reported by the kernel

Kernel messages included:

<span>DMAR: ERROR: DMA PTE for vPFN already set</span>
followed by traces involving:

<span>intel_iommu_map_pages<br>iommu_dma_map_sg<br>scsi_dma_map<br>megasas_build_and_issue_cmd_fusion</span>
and later:

<span>megaraid_sas: resetting fusion adapter scsi0</span>
Interestingly, the issue was observed across multiple servers with nearly identical hardware after the kernel upgrade.

As a mitigation we added:

<span>intel_iommu=on iommu=pt</span>
to the kernel command line.

Since applying this change:

• No new DMA PTE errors have been observed
• No new MegaRAID controller resets have been observed
• Clusters have remained stable

Has anyone experienced similar behavior with:

• Dell R350 (or similar 15G Dell servers)
• PERC H345 / MegaRAID controllers
• Proxmox 8.x kernels in the 6.8 series
• Intel IOMMU / VT-d enabled

I'm particularly interested in whether this is a known regression/change in IOMMU behavior, a MegaRAID driver issue, or a firmware interaction with Dell RMRR/DMAR tables.

Any feedback or similar experiences would be appreciated.

 

[fabian]

please upgrade to the -30 kernel, the -29 one had an IOMMU related regression that is fixed there.

 

[leesteken]

intel_iommu=on is no longer necessary since it is on by default since kernel version 6.8: https://pve.proxmox.com/pve-docs-8/pve-admin-guide.html#_configuration_14
iommu=pt might indeed help non-passedthrough hardware that cannot handle IOMMU very well as it tells the IOMMU to use the identity mapping. There have been more threads on this forum about some RAID controllers having issues with the IOMMU on by default. Newer kernel versions might have driver fixes for such issues.

 

[tycjan]

please upgrade to the -30 kernel, the -29 one had an IOMMU related regression that is fixed there.

We will try to upgrade it once again and have a look.

 

[tycjan]

intel_iommu=on is no longer necessary since it is on by default since kernel version 6.8: https://pve.proxmox.com/pve-docs-8/pve-admin-guide.html#_configuration_14
iommu=pt might indeed help non-passedthrough hardware that cannot handle IOMMU very well as it tells the IOMMU to use the identity mapping. There have been more threads on this forum about some RAID controllers having issues with the IOMMU on by default. Newer kernel versions might have driver fixes for such issues.

Thx, I will have a look.

 

[iss-integration]

I can confirm that we are seeing a similar issue on the latest Proxmox 9 in our lab. The fix so far was to disable IOMMU since we're not doing any pass-through in that particular cluster. iommu=pt still fails, so we had to set intel_iommu=off

Here is a sampling of the messages:

sd ... [sdb] tag#640 OCR is requested due to IO timeout!!
sd ... [sdb] SCSI host state: 5 SCSI host busy: 5 FW outstanding: 5
megaraid_sas 0000:02:00.0: megasas_disable_intr_fusion ...
megaraid_sas 0000:02:00.0: [ 0]waiting for 5 commands to complete for scsi0
... [ 5] ... [10] ... [15] ... [20] ... [25] ... [30]waiting for 5 commands to complete

pveversion
pve-manager/9.2.3/d0fde103346cf89a (running kernel: 7.0.12-1-pve)