[SOLVED] IOMMU issue in old Haswell NUC

[neptune]

Edit: Just needed a BIOS update... See last post.


Hi all!

Does anyone have experience with pcie passthrough on the D34010WYK Intel NUC? It's a Haswell 4010U, 16GB ram. I'm trying to pass an Intel pcie dual NIC to an opnsense VM. I'm not certain but it seems like the grouping should be conducive.

I've tried everything I can think of and always end up with a "No IOMMU detected" message on the hardware pane. Other shell tests not showing it either. I have VT-d and VT-x enabled in the bios. The correct grub options and modules loading per the Proxmox doc on passthrough. Intel's web pages for both the CPU and the NUC as a whole show VT-d is supported.

I have some experience with this on my other nodes, raptor lake and coffee lake platforms. No issues following the same steps for a Coral TPU or Intel NIC.

Any experience using IOMMU on older haswell?

 

[leesteken]

The D34010WYK appears to support VT-d and if you did enable it in the BIOS, then the kernel parameter is probably not active. What is the output of cat /proc/cmdline. Please note that not all Proxmox installations use GRUB.

 

[neptune]

The D34010WYK appears to support VT-d and if you did enable it in the BIOS, then the kernel parameter is probably not active. What is the output of cat /proc/cmdline. Please note that not all Proxmox installations use GRUB.

Should be using grub. Cmdline is BOOT_IMAGE=/boot/vmlinuz-5.15.102-1-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on

 

[leesteken]

Should be using grub. Cmdline is BOOT_IMAGE=/boot/vmlinuz-5.15.102-1-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on

Then VT-d is not (fully) enabled in the BIOS, please double check again. What is the output of journalctl -b | grep iommu?

 

[neptune]

root@vpn:~# journalctl -b | grep iommu
May 29 23:18:30 vpn kernel: Command line: BOOT_IMAGE=/boot/vmlinuz-5.15.102-1-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on
May 29 23:18:30 vpn kernel: Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.15.102-1-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on
May 29 23:18:30 vpn kernel: iommu: Default domain type: Translated
May 29 23:18:30 vpn kernel: iommu: DMA domain TLB invalidation policy: lazy mode

In BIOS VT-x and VT-d are separate options and both checked.

 

[leesteken]

root@vpn:~# journalctl -b | grep iommu
May 29 23:18:30 vpn kernel: Command line: BOOT_IMAGE=/boot/vmlinuz-5.15.102-1-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on
May 29 23:18:30 vpn kernel: Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.15.102-1-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on
May 29 23:18:30 vpn kernel: iommu: Default domain type: Translated
May 29 23:18:30 vpn kernel: iommu: DMA domain TLB invalidation policy: lazy mode

Devices are not assigned to IOMMU groups, otherwise you would see things like kernel: pci 0000:00:01.0: Adding to iommu group 0. Therefore, VT-d is not fully enabled. Adding intel_iommu=on should be enough on the kernel side of things, so it's a hardware/BIOS issue.

In BIOS VT-x and VT-d are separate options and both checked.

But it is not working. I don't know how to fix that, sorry. Maybe you need to enable additional settings like IOMMU or ACS or AER? Maybe a BIOS update? Maybe Intel support knows? Can you find anyone on the internet with the same system who did get it working? Maybe someone else here knows?

 

[neptune]

But it is not working. I don't know how to fix that, sorry. Maybe you need to enable additional settings like IOMMU or ACS or AER? Maybe a BIOS update? Maybe Intel support knows? Can you find anyone on the internet with the same system who did get it working? Maybe someone else here knows?

I haven't been able to yet. Just people mentioning it had the feature in the context of ESXi. Was hoping between here and reddit someone had used it. I'll poke around in the BIOS again tonight. Thanks for the pointers!

 

[neptune]

Well, I feel like an absolute idiot.. All it needed was a BIOS update. Intel makes it oddly hard to find these, they aren't on the main drivers and downloads page for the computer. Stick it on a flash drive and press F7 at boot.

Thank you!!

BIOS update here: https://www.intel.com/content/www/us/en/download/17536/bios-update-wylpt10h.html?v=t

Good way to test if IOMMU is actually working:

root@vpn:~# dmesg | grep "DMAR: Intel"
[    0.445421] DMAR: Intel(R) Virtualization Technology for Directed I/O