Is my IOMMU enabled?

[Hanan]

Hi,

I have the following:
[ 0.136023] AMD-Vi: ivrs, add hidNP0D40, uid:, rdevid:152
[ 0.378700] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[ 0.379902] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 0.379903] AMD-Vi: Extended features (0x4f77ef22294ada): PPR NX GT IA GA PC GA_vAPIC
[ 0.379910] AMD-Vi: Interrupt remapping enabled
[ 0.379911] AMD-Vi: Virtual APIC enabled
[ 0.380139] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).
[ 4.851855] AMD-Vi: AMD IOMMUv2 loaded and initialized

However i dont actually see "DMAR: IOMMU enable"

Do i have IOMMU enabled?
I am unable to start a VM with pci passthrough at the moment

 

[leesteken]

However i dont actually see "DMAR: IOMMU enable"

Looks like you use a test for Intel systems and you appear to have an AMD system. Please share the make and model of your motherboard.

Do i have IOMMU enabled?

On (recent) AMD systems IOMMU is enabled by default, so most likely yes. If there is more than one IOMMU group, it's enabled. Use this command to see your IOMMU groups:
for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done

I am unable to start a VM with pci passthrough at the moment

How does "unable to start" manifest itself to you? Any error messages in the Task log or Syslog (or journalctl)?
Please show us the whole( IOMMU group (with the command above) for the devices you want to passthrough and the VM configuration file (from the /etc/pve/qemu-server/ directory) (with the passthrough).

 

[Hanan]

Looks like you use a test for Intel systems and you appear to have an AMD system. Please share the make and model of your motherboard.

On (recent) AMD systems IOMMU is enabled by default, so most likely yes. If there is more than one IOMMU group, it's enabled. Use this command to see your IOMMU groups:
for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done

How does "unable to start" manifest itself to you? Any error messages in the Task log or Syslog (or journalctl)?
Please show us the whole( IOMMU group (with the command above) for the devices you want to passthrough and the VM configuration file (from the /etc/pve/qemu-server/ directory) (with the passthrough).

Thanks for the help....note my question was "is my iommu" enabled thats why i didn't provide much details on my VM not starting....

can we maybe start with that (VM has been deleted for now)
I am still not sure what the answer is....

I do have an AMD system.
AMD Athlon Silver 3050e, not too sure about the motherboard.

 

[Hanan]

IOMMU group 0 00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU group 10 00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 0 [1022:15e8]
IOMMU group 10 00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 1 [1022:15e9]
IOMMU group 10 00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 2 [1022:15ea]
IOMMU group 10 00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 3 [1022:15eb]
IOMMU group 10 00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 4 [1022:15ec]
IOMMU group 10 00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 5 [1022:15ed]
IOMMU group 10 00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 6 [1022:15ee]
IOMMU group 10 00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 7 [1022:15ef]
IOMMU group 11 01:00.0 Non-Volatile memory controller [0108]: Kingston Technology Company, Inc. OM3PDP3 NVMe SSD [2646:500d] (rev 01)
IOMMU group 12 02:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 0c)
IOMMU group 13 03:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 0c)
IOMMU group 14 04:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 0c)
IOMMU group 15 05:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 0c)
IOMMU group 16 06:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Picasso [1002:15d8] (rev de)
IOMMU group 17 06:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Raven/Raven2/Fenghuang HDMI/DP Audio Controller [1002:15de]
IOMMU group 17 06:00.2 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) Platform Security Processor [1022:15df]
IOMMU group 17 06:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Raven2 USB 3.1 [1022:15e5]
IOMMU group 17 06:00.5 Multimedia controller [0480]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2/FireFlight/Renoir Audio Processor [1022:15e2]
IOMMU group 17 06:00.6 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) HD Audio Controller [1022:15e3]
IOMMU group 17 06:00.7 Non-VGA unclassified device [0000]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2/Renoir Non-Sensor Fusion Hub KMDF driver [1022:15e6]
IOMMU group 18 07:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 61)
IOMMU group 1 00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 PCIe GPP Bridge [6:0] [1022:15d3]
IOMMU group 2 00:01.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 PCIe GPP Bridge [6:0] [1022:15d3]
IOMMU group 3 00:01.3 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 PCIe GPP Bridge [6:0] [1022:15d3]
IOMMU group 4 00:01.4 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 PCIe GPP Bridge [6:0] [1022:15d3]
IOMMU group 5 00:01.5 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 PCIe GPP Bridge [6:0] [1022:15d3]
IOMMU group 6 00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU group 7 00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Internal PCIe GPP Bridge 0 to Bus A [1022:15db]
IOMMU group 8 00:08.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Internal PCIe GPP Bridge 0 to Bus B [1022:15dc]
IOMMU group 9 00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 61)
IOMMU group 9 00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)

 

[leesteken]

Thanks for the help....note my question was "is my iommu" enabled thats why i didn't provide much details on my VM not starting....

No problem, I just wanted to tell you what kind of information I need to look into a non-starting VM with passthrough..

can we maybe start with that (VM has been deleted for now)

Sure no problem. But I do need some more information to answer your question.

I am still not sure what the answer is....

Neither am I because I need the (whole) output of for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done. That is the ultimate test of whether IOMMU is enabled.

AMD Athlon Silver 3050e, not too sure about the motherboard.

That appears to be a CPU for a laptop and I can understand that the motherboard is unclear. It's a Raven Ridge Ryzen CPU and I expect that it supports IOMMU but I cannot be sure about whether your laptop supports it or whether it is enabled. Please share the output of the command above and I can tell you the answer.

 

[Hanan]

I did log into the bios and changed IOMMU from auto to enabled, so on the bios part i know it's supported and enabled.

 

[Hanan]

I did upload the command...
but "This message is awaiting moderator approval, and is invisible to normal visitors. "

 

[Hanan]

alright i lowered the RAM allocation for the VM and its working....strange?
I have 16GB and proxmox showing 13.61....is this due to the IOMMU?

 

[leesteken]

I did log into the bios and changed IOMMU from auto to enabled, so on the bios part i know it's supported and enabled.

Glad to see that you got it working.

I did upload the command...
but "This message is awaiting moderator approval, and is invisible to normal visitors. "

My apologies. I expected the output of the command in the same message (and did not see the invisible one).

alright i lowered the RAM allocation for the VM and its working....strange?

PCI(e) devices can do DMA to and from any part of the memory at any time. Therefore all VM memory must be pinned into actual RAM on the Proxmox host. Sometimes there is not enough continuous memory available and this typically results in a time-out without further messages (that's why I asked before). For the same reason ballooning does not work for a VM with passthrough. If you remove the passthrough, then the VM might start but eventually run into issues when your Proxmox host cannot provide all the memory at once.

I have 16GB and proxmox showing 13.61....is this due to the IOMMU?

Your laptop has integrated graphics that also need memory and the Linux kernel also needs some memory to manage the rest of the memory. My guess is the kernel reserved memory is about 1.3GiB or so (similar to my system) and you allocated 1GiB (in the BIOS or maybe that's the default set by Linux) for the integrated graphics. This is normal and does not depend on IOMMU.

 

[Hanan]

Thanks for all the help.
by the way, this is not a laptop (but yes definitely laptop components ).
got myself a "Topton 2-Bay nas"

Busy testing if it can replace my hp micro server.
decided to run proxmox with a truenas VM.

 

[Argiros]

Hi.
I'm looking oy buy a Topton 2-Bays too. Run proxmox with SATA bypass to a NAS VM (true nas, unpaid, etc).
How are you experience after these month with this system?

Thank you!

 

[Bobbbb]

I am going to get something more powerful.
not too sure if sata passthrough is possible but i am giving up for now at least.

I think i was messing around too much and at some point my hard drives were not detected, I thought it was the topon but after reloading proxmox it's fine again, but the sata controller is proving to be tricky.
https://forum.proxmox.com/posts/531582/

 

[gilfreund]

Hi.
I'm looking oy buy a Topton 2-Bays too. Run proxmox with SATA bypass to a NAS VM (true nas, unpaid, etc).
How are you experience after these month with this system?

Thank you!

I am using this system with Virtualized TrueNAS inside Proxmox. I did not manage to bypass the SATA controllers, but I did manage to pass the actual disks, and it works fine (did not do any benchmarks, though)