[SOLVED] DMA read error on Marvell SATA controller

[blackjacktheripper]

SYSTEM:
My system is an HPE ML350 G6 using a Xeon 5645 running ProxmoxVE 6.3-3. VT-d enabled in BIOS and IOMMU enabled in GRUB settings.
As my system is affected by the rmrr-issues, my kernel has the fix patched in: https://github.com/kiler129/relax-intel-rmrr/blob/master/README.md#proxmox---building-from-sources which enables me to pass PCI(e) devices successfully to VMs.
I have two SATA controllers installed (because the onboards are very slow) which use the Marvell 9215 controller with one being connected to my Seagate HDD.

PROBLEM:
The problem is twofold:
- when IOMMU is enabled, DMA read errors are thrown, the controller shows up in lscpi but drives are not detected
- when passing on the controller (to a recent FreeBSD/truenas), cam-errors are encountered during boot, controller will be detected but no drives
The problem exactly matches the one described in this bugzilla thread: https://bugzilla.kernel.org/show_bug.cgi?id=42679

NEED HELP WITH:
I have very limited experience with compiling kernels. I used the pre-built kernel from the rmrr-fix github.
There seems to be a patch available but I am not sure whether that is implemented in the version of the Linux kernel that is used by proxmox. I also do not know how to apply the patch. Additionally there was a recent post to the bugzilla thread, stating that adding something to a quirks file resolved this problem with the same controller I have. As I understand it the patch is only required in the host, correct me if I am wrong.
tl;dr please tell me how to apply the patch or where to modify the kernel to include my controller in it.

Thank you for your time. If any logs are required (I have not posted them here as the problem seems to have been diagnosed to be an issue with the Marvell controller) feel free to ask.

UPDATE 01:
I have compiled a kernel including the rmrr patch (through it, I can verify the kernel actually works, no rmrr errors) and a custom patch to the file drivers/pci/quirks.c. As you can see, I have added a logging output which should be visible in dmesg when the fix is started but as evident by the output of dmesg | grep -e "DMAR" the fix is never applied and the issue persists. I sadly do not see the mistake here and the bugzilla board suggests that this fix with the line copied 1:1 from someone using the exact chipset as I am can make it work.

UPDATE 02:
As a troubleshooting step, I directly edited the quirks.c file and after rebuilding the kernel got the logging output. The drive also did show up in the proxmox disk list. Passing the controller through to the FreeBSD guest also works now. Technically solved but I'd love some input on what I did wrong with the patch that caused it to not work or whether it should have worked and I just missed a step somewhere.

SOLUTION:
Gather information on how to create a custom patch using this forum article https://forum.proxmox.com/threads/c...ommu-driver-to-remove-rmrr-check.36374/page-1
Then determine the model number of your chip, mine is 9125, and consult https://bugzilla.kernel.org/show_bug.cgi?id=42679 on what code has to be changed. With the previously gathered information one can now build a patch for the drivers/pci/quirks.c file and compile a kernel with said patch. My patch is appended here but as I stated previously, it does not work and I had to manually make these changes in the file.
I will not yet mark this as solved because manually editing the file does not seem like a good solution and it would be very nice to receive some comment on why my patch does not work as expected.

 

[leesteken]

I'm sorry but I can't really help you with your actual questions. If you have the time, could you please check if the DMA errors on the host go away then you add iommu=pt to the kernel paramters (but don't remove intel_iommu)? This will probably not help when you want to passthrough the device, but I wonder if it helps when people use that SATA controller on the host. Feel free to ignore this request, I'm just trying to validate my knowledge of IOMMU and it will probably not help you.

 

[blackjacktheripper]

I'm sorry but I can't really help you with your actual questions. If you have the time, could you please check if the DMA errors on the host go away then you add iommu=pt to the kernel paramters (but don't remove intel_iommu)? This will probably not help when you want to passthrough the device, but I wonder if it helps when people use that SATA controller on the host. Feel free to ignore this request, I'm just trying to validate my knowledge of IOMMU and it will probably not help you.

Compiling a kernel (with hopefully working fix atm). I had iommu=pt in my grub parameters and the issue persisted, however I do not remember if I ever checked the logs with the controller not passed through. I will update this once the server is done compiling.
Here the update: As suspected, setting iommu=pt does not prevent the errors from popping up. Your understanding of this option is correct in that it will only enable IOMMU for devices passed through. I do not understand why the controller behaves this way but reading through the bugzilla article I found that to replicate this issue, the device itself mustnt neccessarily be passed on but just enabled.