Enabling IOMMU on AMD/Asus Desktop

[padekan]

Hi,

I have the following system set up:
- Asus M11BB System
- F2A55-M Motherboard
- AMD A8-6500 APU
- 32 GB DRAM
- Two SATA 256GB SSDs on the motherboard SATA connection (local disks for Proxmox & containers)
- Two SATA 1TB SSDs on a PCIE Gen 2 16x1 adapter (which can host up to 6 drives)
- On the ASUS UEFI Bios there is only an SVM Enabled (to allow virtualization)

Proxmox installed fine. I have Pi-hole and Homebridge containers which are operating fine
I want to install TrueNas Core as a VM and that is I ran into problems trying to pass through the PCIE adapter to the VM (error says IOMMU is not enabled)

I have tried modifying /etc/default/group to add the "and_iommu=on iommu=pt" but that doesn't seem to have any effect (the dmesg looking of DMAR or IOMMU doesn't show anything). I've also added the 4 vfio entries to /etc/modules and reinitialized the ramdisk

I got the CPU data sheet from AMD and it shows it should support AMD-V as well as IOMMU 2.0. I also downloaded the ASUS manual for the M11BB and it shows there should be an IOMMU enable setting in the BIOS (in their screen shots). However, that doesn't show up in my menus. Interesting and possibly unrelated I also don;t have the "tinker" tab in the BIOS that allows you to play around with overclocking.

I expect either my motherboard was somehow deprecated (but I can't see any jumpers). So I'm looking for options .. ideally if I could just give the second SATA adapter to the TrueNas VM without Proxmox knowing about it, that would be good. Or if someone knows how to hijack my BIOS to turn on IOMMU, that would be great.

Any insights are appreciated!

ps. I know it's an 11 year old desktop but it actually does a fine job running TrueNas on bare metal ... but I had trouble configuring VMs on that so I went to Proxmox.

 

[rtorres]

Do you boot using grub or the systemd?

For grub:
nano /etc/default/grub
it should look like this: GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on"
update-grub

For systemd:
nano /etc/kernel/cmdline
add 'amd_iommu=on'
proxmox-boot-tool refresh

Then:
nano /etc/modules
add:
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

Apply changes (both grub & systemd boot)
update-initramfs -u -k all

Then reboot the PC.

 

[leesteken]

amd_iommu=on does nothing (and is not even valid) since it is enable by default.
AMD-V is for virtualization like running VMs but you need AMD-Vi (notice the little i at the end) for IOMMU and PCI(e) passthrough support.
I'm almost sure that your FM2 socket CPU (from 2013) does not support AMD-Vi: https://www.amd.com/en/support/down...or-desktops/a8-6500-with-radeon-hd-8570d.html

 

[padekan]

amd_iommu=on does nothing (and is not even valid) since it is enable by default.
AMD-V is for virtualization like running VMs but you need AMD-Vi (notice the little i at the end) for IOMMU and PCI(e) passthrough support.
I'm almost sure that your FM2 socket CPU (from 2013) does not support AMD-Vi: https://www.amd.com/en/support/down...or-desktops/a8-6500-with-radeon-hd-8570d.html

HI,

Thanks for the link .. that is where I found this:


which does spect to IOMMU. Also, the motherboard user manual has a bios screen shot which shows IOMMU Enable as a setting (but it is not in the bios on my machine)

 

[leesteken]

View attachment 71564
which does spect to IOMMU. Also, the motherboard user manual has a bios screen shot which shows IOMMU Enable as a setting (but it is not in the bios on my machine)

I'm still worried about the lack of AMD-Vi mentioned there but maybe ask the manufacturer for a BIOS update or support for turning IOMMU on. If it's not enabled by the motherboard BIOS then Proxmox (or Linux) cannot do anything to enable it. Or maybe try an older BIOS version that still has the IOMMU setting?
Please note take flashing older (or newer) BIOS version might cause problems or brick the system if interrupted. Make sure to clear CMOS and Load Optimized Defaults after flashing.

 

[padekan]

Do you boot using grub or the systemd?

For grub:
nano /etc/default/grub
it should look like this: GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on"
update-grub

For systemd:
nano /etc/kernel/cmdline
add 'amd_iommu=on'
proxmox-boot-tool refresh

Then:
nano /etc/modules
add:
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

Apply changes (both grub & systemd boot)
update-initramfs -u -k all

Then reboot the PC.

Hi,

Thanks for the quick reply .. I redid all the grub changes per your you process but nothing changed.

I also tried setting the parameters in /etc/kernel/cmdline and ran proxmox-boot-tool refresh

A new message showed up in dmesg "iommu: Default domain type: Passthrough (set via kernel command line)" but Proxmox still says IOMMU is not enabled.

Any other guidance is greatly appreciated.

 

[leesteken]

Thanks for the quick reply .. I redid all the grub changes per your you process but nothing changed.

I also tried setting the parameters in /etc/kernel/cmdline and ran proxmox-boot-tool refresh

A new message showed up in dmesg "iommu: Default domain type: Passthrough (set via kernel command line)" but Proxmox still says IOMMU is not enabled.

Those things you did have no influence at all when it is not enabled (or support) in the motherboard BIOS. I'm sorry if I gave the impression that it might change things.

Any other guidance is greatly appreciated.

As I said, try other BIOS versions, contact the manufacturer, search the internet for other peoples experiences with this board and IOMMU/AMD-Vi.