Enabling IOMMU in Proxmox VE 8 Kernel

D

doremo

New Member
Apr 11, 2024
11
1
3
Do we still need to manually enable IOMMU in the kernel for Proxmox VE 8 by setting intel_iommu=on and intel_iommu=pt?
 
leesteken said:
There is no such thing as intel_iommu=pt and yes intel_iommu=on is needed: https://pve.proxmox.com/pve-docs/pve-admin-guide.html#qm_pci_passthrough

EDIT: And there are threads here about this twice a week, if you need more detailed information.
Click to expand...
I believe IOMMU is enabled by default on my Proxmox VE 8 system. I observed that the outputs of dmesg | grep -i iommu were identical before and after adding intel_iommu=on to the kernel parameters, indicating no change in the IOMMU settings.
 
doremo said:
I believe IOMMU is enabled by default on my Proxmox VE 8 system.
Click to expand...
That is only when you use an AMD CPU. What CPU and motherboard are you using?
doremo said:
I observed that the outputs of dmesg | grep -i iommu were identical before and after adding intel_iommu=on to the kernel parameters, indicating no change in the IOMMU settings.
Click to expand...
It is quite possible you added it in the wrong place (or VT-d is not enabled or not supported) and it did not make a change (or you are using an AMD CPU).
What is the output of cat /proc/cmdline and for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done?

EDIT: Intel IOMMU was enabled by default on PVE 7 for a little while but it broke too many Intel systems and they reverted it back (and updated the manual years later).
 
Last edited:
leesteken said:
That is only when you use an AMD CPU. What CPU and motherboard are you using?

It is quite possible you added it in the wrong place (or VT-d is not enabled or not supported) and it did not make a change (or you are using an AMD CPU).
What is the output of cat /proc/cmdline and for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done?

EDIT: Intel IOMMU was enabled by default on PVE 7 for a little while but it broke too many Intel systems and they reverted it back (and updated the manual years later).
Click to expand...
I am SI engineer so I know how to differential AMD and Intel, thanks
 
doremo said:
I am SI engineer so I know how to differential AMD and Intel, thanks
Click to expand...
I'm glad that IOMMU worked by default on your system but many people did need to select a motherboard and Intel CPU that supports VT-d, enable it in BIOS and add intel_iommu=on to the PVE8 kernel parameters before IOMMU works (as a search on this forum will show).
 
leesteken said:
That is only when you use an AMD CPU. What CPU and motherboard are you using?

It is quite possible you added it in the wrong place (or VT-d is not enabled or not supported) and it did not make a change (or you are using an AMD CPU).
What is the output of cat /proc/cmdline and for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done?

EDIT: Intel IOMMU was enabled by default on PVE 7 for a little while but it broke too many Intel systems and they reverted it back (and updated the manual years later).
Click to expand...
root@pve:~# cat /proc/cmdline
for d in /sys/kernel/iommu_groups/*/devices/*; do
n=${d#*/iommu_groups/*}; n=${n%%/*};
printf 'IOMMU group %s ' "$n";
lspci -nns "${d##*/}";
done
initrd=\EFI\proxmox\6.5.13-5-pve\initrd.img-6.5.13-5-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs
IOMMU group 0 0000:00:02.0 VGA compatible controller [0300]: Intel Corporation Raptor Lake-P [Iris Xe Graphics] [8086:a7a0] (rev 04)
IOMMU group 10 0000:00:16.0 Communication controller [0780]: Intel Corporation Alder Lake PCH HECI Controller [8086:51e0] (rev 01)
IOMMU group 10 0000:00:16.3 Serial controller [0700]: Intel Corporation Alder Lake AMT SOL Redirection [8086:51e3] (rev 01)
IOMMU group 11 0000:00:1c.0 PCI bridge [0604]: Intel Corporation Alder Lake-P PCH PCIe Root Port [8086:51bb] (rev 01)
IOMMU group 12 0000:00:1d.0 System peripheral [0880]: Intel Corporation RST VMD Managed Controller [8086:09ab]
IOMMU group 13 0000:00:1d.2 PCI bridge [0604]: Intel Corporation Device [8086:51b2] (rev 01)
IOMMU group 14 0000:00:1d.3 PCI bridge [0604]: Intel Corporation Device [8086:51b3] (rev 01)
IOMMU group 15 0000:00:1f.0 ISA bridge [0601]: Intel Corporation Raptor Lake LPC/eSPI Controller [8086:519d] (rev 01)
IOMMU group 15 0000:00:1f.3 Audio device [0403]: Intel Corporation Raptor Lake-P/U/H cAVS [8086:51ca] (rev 01)
IOMMU group 15 0000:00:1f.4 SMBus [0c05]: Intel Corporation Alder Lake PCH-P SMBus Host Controller [8086:51a3] (rev 01)
IOMMU group 15 0000:00:1f.5 Serial bus controller [0c80]: Intel Corporation Alder Lake-P PCH SPI Controller [8086:51a4] (rev 01)
IOMMU group 16 0000:01:00.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1812 6-Port PCIe x4 Gen2 Packet Switch [1b21:1812] (rev 01)
IOMMU group 17 0000:02:00.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1812 6-Port PCIe x4 Gen2 Packet Switch [1b21:1812] (rev 01)
IOMMU group 17 0000:03:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 05)
IOMMU group 18 0000:02:08.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1812 6-Port PCIe x4 Gen2 Packet Switch [1b21:1812] (rev 01)
IOMMU group 18 0000:04:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 05)
IOMMU group 19 0000:02:0a.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1812 6-Port PCIe x4 Gen2 Packet Switch [1b21:1812] (rev 01)
IOMMU group 19 0000:05:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 05)
IOMMU group 1 0000:00:00.0 Host bridge [0600]: Intel Corporation Device [8086:a706]
IOMMU group 20 0000:02:0b.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1812 6-Port PCIe x4 Gen2 Packet Switch [1b21:1812] (rev 01)
IOMMU group 20 0000:06:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 05)
IOMMU group 21 0000:07:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ [8086:1572] (rev 02)
IOMMU group 22 0000:07:00.1 Ethernet controller [0200]: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ [8086:1572] (rev 02)
IOMMU group 23 0000:5c:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller I226-V [8086:125c] (rev 04)
IOMMU group 24 0000:5d:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller I226-LM [8086:125b] (rev 04)
IOMMU group 25 0000:5e:00.0 Network controller [0280]: MEDIATEK Corp. MT7922 802.11ax PCI Express Wireless Network Adapter [14c3:0616]
IOMMU group 2 0000:00:01.0 PCI bridge [0604]: Intel Corporation Device [8086:a70d]
IOMMU group 3 0000:00:06.0 System peripheral [0880]: Intel Corporation RST VMD Managed Controller [8086:09ab]
IOMMU group 4 0000:00:06.2 PCI bridge [0604]: Intel Corporation Device [8086:a73d]
IOMMU group 5 0000:00:07.0 PCI bridge [0604]: Intel Corporation Raptor Lake-P Thunderbolt 4 PCI Express Root Port [8086:a76e]
IOMMU group 6 0000:00:07.2 PCI bridge [0604]: Intel Corporation Raptor Lake-P Thunderbolt 4 PCI Express Root Port [8086:a72f]
IOMMU group 7 0000:00:0d.0 USB controller [0c03]: Intel Corporation Raptor Lake-P Thunderbolt 4 USB Controller [8086:a71e]
IOMMU group 7 0000:00:0d.2 USB controller [0c03]: Intel Corporation Raptor Lake-P Thunderbolt 4 NHI [8086:a73e]
IOMMU group 7 0000:00:0d.3 USB controller [0c03]: Intel Corporation Raptor Lake-P Thunderbolt 4 NHI [8086:a76d]
IOMMU group 8 0000:00:0e.0 RAID bus controller [0104]: Intel Corporation Volume Management Device NVMe RAID Controller Intel Corporation [8086:a77f]
IOMMU group 8 10000:e0:06.0 PCI bridge [0604]: Intel Corporation Raptor Lake PCIe 4.0 Graphics Port [8086:a74d]
IOMMU group 8 10000:e0:1c.0 System peripheral [0880]: Intel Corporation RST VMD Managed Controller [8086:09ab]
IOMMU group 8 10000:e0:1c.4 PCI bridge [0604]: Intel Corporation Device [8086:51bc] (rev 01)
IOMMU group 8 10000:e0:1d.0 PCI bridge [0604]: Intel Corporation Alder Lake PCI Express Root Port [8086:51b0] (rev 01)
IOMMU group 8 10000:e1:00.0 Non-Volatile memory controller [0108]: Phison Electronics Corporation PS5019-E19 PCIe4 NVMe Controller (DRAM-less) [1987:5019] (rev 01)
IOMMU group 8 10000:e2:00.0 Non-Volatile memory controller [0108]: Phison Electronics Corporation PS5019-E19 PCIe4 NVMe Controller (DRAM-less) [1987:5019] (rev 01)
IOMMU group 8 10000:e3:00.0 Non-Volatile memory controller [0108]: Phison Electronics Corporation PS5019-E19 PCIe4 NVMe Controller (DRAM-less) [1987:5019] (rev 01)
IOMMU group 9 0000:00:14.0 USB controller [0c03]: Intel Corporation Alder Lake PCH USB 3.2 xHCI Host Controller [8086:51ed] (rev 01)
IOMMU group 9 0000:00:14.2 RAM memory [0500]: Intel Corporation Alder Lake PCH Shared SRAM [8086:51ef] (rev 01)
 
  • Like
Reactions: leesteken
doremo said:
initrd=\EFI\proxmox\6.5.13-5-pve\initrd.img-6.5.13-5-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs
IOMMU group 0 0000:00:02.0 VGA compatible controller [0300]: Intel Corporation Raptor Lake-P [Iris Xe Graphics] [8086:a7a0] (rev 04)
IOMMU group 10 0000:00:16.0 Communication controller [0780]: Intel Corporation Alder Lake PCH HECI Controller [8086:51e0] (rev 01)
IOMMU group 10 0000:00:16.3 Serial controller [0700]: Intel Corporation Alder Lake AMT SOL Redirection [8086:51e3] (rev 01)
IOMMU group 11 0000:00:1c.0 PCI bridge [0604]: Intel Corporation Alder Lake-P PCH PCIe Root Port [8086:51bb] (rev 01)
IOMMU group 12 0000:00:1d.0 System peripheral [0880]: Intel Corporation RST VMD Managed Controller [8086:09ab]
IOMMU group 13 0000:00:1d.2 PCI bridge [0604]: Intel Corporation Device [8086:51b2] (rev 01)
IOMMU group 14 0000:00:1d.3 PCI bridge [0604]: Intel Corporation Device [8086:51b3] (rev 01)
IOMMU group 15 0000:00:1f.0 ISA bridge [0601]: Intel Corporation Raptor Lake LPC/eSPI Controller [8086:519d] (rev 01)
IOMMU group 15 0000:00:1f.3 Audio device [0403]: Intel Corporation Raptor Lake-P/U/H cAVS [8086:51ca] (rev 01)
IOMMU group 15 0000:00:1f.4 SMBus [0c05]: Intel Corporation Alder Lake PCH-P SMBus Host Controller [8086:51a3] (rev 01)
IOMMU group 15 0000:00:1f.5 Serial bus controller [0c80]: Intel Corporation Alder Lake-P PCH SPI Controller [8086:51a4] (rev 01)
IOMMU group 16 0000:01:00.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1812 6-Port PCIe x4 Gen2 Packet Switch [1b21:1812] (rev 01)
IOMMU group 17 0000:02:00.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1812 6-Port PCIe x4 Gen2 Packet Switch [1b21:1812] (rev 01)
IOMMU group 17 0000:03:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 05)
IOMMU group 18 0000:02:08.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1812 6-Port PCIe x4 Gen2 Packet Switch [1b21:1812] (rev 01)
IOMMU group 18 0000:04:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 05)
IOMMU group 19 0000:02:0a.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1812 6-Port PCIe x4 Gen2 Packet Switch [1b21:1812] (rev 01)
IOMMU group 19 0000:05:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 05)
IOMMU group 1 0000:00:00.0 Host bridge [0600]: Intel Corporation Device [8086:a706]
IOMMU group 20 0000:02:0b.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1812 6-Port PCIe x4 Gen2 Packet Switch [1b21:1812] (rev 01)
IOMMU group 20 0000:06:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 05)
IOMMU group 21 0000:07:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ [8086:1572] (rev 02)
IOMMU group 22 0000:07:00.1 Ethernet controller [0200]: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ [8086:1572] (rev 02)
IOMMU group 23 0000:5c:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller I226-V [8086:125c] (rev 04)
IOMMU group 24 0000:5d:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller I226-LM [8086:125b] (rev 04)
IOMMU group 25 0000:5e:00.0 Network controller [0280]: MEDIATEK Corp. MT7922 802.11ax PCI Express Wireless Network Adapter [14c3:0616]
IOMMU group 2 0000:00:01.0 PCI bridge [0604]: Intel Corporation Device [8086:a70d]
IOMMU group 3 0000:00:06.0 System peripheral [0880]: Intel Corporation RST VMD Managed Controller [8086:09ab]
IOMMU group 4 0000:00:06.2 PCI bridge [0604]: Intel Corporation Device [8086:a73d]
IOMMU group 5 0000:00:07.0 PCI bridge [0604]: Intel Corporation Raptor Lake-P Thunderbolt 4 PCI Express Root Port [8086:a76e]
IOMMU group 6 0000:00:07.2 PCI bridge [0604]: Intel Corporation Raptor Lake-P Thunderbolt 4 PCI Express Root Port [8086:a72f]
IOMMU group 7 0000:00:0d.0 USB controller [0c03]: Intel Corporation Raptor Lake-P Thunderbolt 4 USB Controller [8086:a71e]
IOMMU group 7 0000:00:0d.2 USB controller [0c03]: Intel Corporation Raptor Lake-P Thunderbolt 4 NHI [8086:a73e]
IOMMU group 7 0000:00:0d.3 USB controller [0c03]: Intel Corporation Raptor Lake-P Thunderbolt 4 NHI [8086:a76d]
IOMMU group 8 0000:00:0e.0 RAID bus controller [0104]: Intel Corporation Volume Management Device NVMe RAID Controller Intel Corporation [8086:a77f]
IOMMU group 8 10000:e0:06.0 PCI bridge [0604]: Intel Corporation Raptor Lake PCIe 4.0 Graphics Port [8086:a74d]
IOMMU group 8 10000:e0:1c.0 System peripheral [0880]: Intel Corporation RST VMD Managed Controller [8086:09ab]
IOMMU group 8 10000:e0:1c.4 PCI bridge [0604]: Intel Corporation Device [8086:51bc] (rev 01)
IOMMU group 8 10000:e0:1d.0 PCI bridge [0604]: Intel Corporation Alder Lake PCI Express Root Port [8086:51b0] (rev 01)
IOMMU group 8 10000:e1:00.0 Non-Volatile memory controller [0108]: Phison Electronics Corporation PS5019-E19 PCIe4 NVMe Controller (DRAM-less) [1987:5019] (rev 01)
IOMMU group 8 10000:e2:00.0 Non-Volatile memory controller [0108]: Phison Electronics Corporation PS5019-E19 PCIe4 NVMe Controller (DRAM-less) [1987:5019] (rev 01)
IOMMU group 8 10000:e3:00.0 Non-Volatile memory controller [0108]: Phison Electronics Corporation PS5019-E19 PCIe4 NVMe Controller (DRAM-less) [1987:5019] (rev 01)
IOMMU group 9 0000:00:14.0 USB controller [0c03]: Intel Corporation Alder Lake PCH USB 3.2 xHCI Host Controller [8086:51ed] (rev 01)
IOMMU group 9 0000:00:14.2 RAM memory [0500]: Intel Corporation Alder Lake PCH Shared SRAM [8086:51ef] (rev 01)
Click to expand...
Amazing! Did Proxmox enable intel_iommu by default again?!? And without updating the manual (again), after they recently changed it to:
For Intel CPUs, you also need to enable the IOMMU on the kernel command line kernels by adding:
intel_iommu=on
Click to expand...

EDIT: Just the other day people still needed to add intel_iommu to it to work: https://forum.proxmox.com/threads/iommu-groups-on-supermicro-x13sch-f.145392/post-655551
 
Last edited:
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back