[SOLVED] Need help on PCIe passthrough(intel iommu)!

alex01763

New Member
Feb 28, 2023
8
0
1
Hi everyone

As you might have guessed, I'm having trouble with getting pcie passthrough to work. I think I've tried everything I can find on google and youtube, which were mostly the same. Here's my progress so far:

My /etc/default/grub line is: GRUB_CMDLINE_LINUX_DEFAULT="quiet nmi_watchdog=1 intel_iommu_on iommu=pt"
I tried various acs override configurations but to no avail.

When I run the dmsg command in the proxmox manual I get this:
Code:
root@node-main:~/Scripts# dmesg | grep -e DMAR -e IOMMU
[    0.009413] ACPI: DMAR 0x000000008A3EC828 0000A8 (v01 INTEL  EDK2     00000002      01000013)
[    0.009438] ACPI: Reserving DMAR table memory at [mem 0x8a3ec828-0x8a3ec8cf]
[    0.096874] DMAR: Host address width 39
[    0.096875] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[    0.096880] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 1c0000c40660462 ecap 19e2ff0505e
[    0.096883] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[    0.096886] DMAR: dmar1: reg_base_addr fed91000 ver 1:0 cap d2008c40660462 ecap f050da
[    0.096887] DMAR: RMRR base: 0x0000008aaca000 end: 0x0000008ad13fff
[    0.096889] DMAR: RMRR base: 0x0000008b800000 end: 0x0000008fffffff
[    0.096891] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.096892] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[    0.096893] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.098903] DMAR-IR: Enabled IRQ remapping in x2apic mode

I found out on arch linux forum that the last line means vt-d is enabled, if I understand correctly.
But when I run this line I found in this forum:
Code:
root@node-main:~/Scripts# for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU Group %s ' "$n"; lspci -nns "${d##*/}"; done
IOMMU Group * 00:00.0 Host bridge [0600]: Intel Corporation Device [8086:3e0f] (rev 08)
00:02.0 VGA compatible controller [0300]: Intel Corporation CoffeeLake-S GT1 [UHD Graphics 610] [8086:3e93]
00:08.0 System peripheral [0880]: Intel Corporation Xeon E3-1200 v5/v6 / E3-1500 v5 / 6th/7th/8th Gen Core Processor Gaussian Mixture Model [8086:1911]
00:12.0 Signal processing controller [1180]: Intel Corporation Cannon Lake PCH Thermal Controller [8086:a379] (rev 10)
00:14.0 USB controller [0c03]: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller [8086:a36d] (rev 10)
00:14.2 RAM memory [0500]: Intel Corporation Cannon Lake PCH Shared SRAM [8086:a36f] (rev 10)
00:16.0 Communication controller [0780]: Intel Corporation Cannon Lake PCH HECI Controller [8086:a360] (rev 10)
00:17.0 SATA controller [0106]: Intel Corporation Cannon Lake PCH SATA AHCI Controller [8086:a352] (rev 10)
00:1c.0 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port #5 [8086:a33c] (rev f0)
00:1f.0 ISA bridge [0601]: Intel Corporation Device [8086:a303] (rev 10)
00:1f.3 Audio device [0403]: Intel Corporation Cannon Lake PCH cAVS [8086:a348] (rev 10)
00:1f.4 SMBus [0c05]: Intel Corporation Cannon Lake PCH SMBus Controller [8086:a323] (rev 10)
00:1f.5 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH SPI Controller [8086:a324] (rev 10)
01:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 15)

I don't get any IOMMU groups. There aren't any third party gpus visible because I've sent my gtx 960 to get it fixed right now but still, I can't add any pcie hardware to my virtual machines because 'iommu is not enabled'.

I've spent too much time working on it now and I need your help. Thanks in advance!
 
If there are no multiple groups (only *) then IOMMU is not enabled. What is the output of cat /proc/cmdline? Maybe your Proxmox uses systemd-boot instead of GRUB?
I've checked that also, I think it uses grub, but I'm not a linux expert so I'm not a 100% sure. here's the output of cat

root@node-main:~# cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.15.85-1-pve root=/dev/mapper/pve-root ro quiet nmi_watchdog=1 intel_iommu_on iommu=pt crashkernel=384M-:256M
 
I've checked that also, I think it uses grub, but I'm not a linux expert so I'm not a 100% sure. here's the output of cat

root@node-main:~# cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.15.85-1-pve root=/dev/mapper/pve-root ro quiet nmi_watchdog=1 intel_iommu_on iommu=pt crashkernel=384M-:256M
I agree, it is using GRUB. However, the correct way to enable IOMMU on Intel is with intel_iommu=on and not what you have currently written in GRUB. The second underscore is a typing mistake and should be an equals sign.
 
  • Like
Reactions: alex01763
I agree, it is using GRUB. However, the correct way to enable IOMMU on Intel is with intel_iommu=on and not what you have currently written in GRUB. The second underscore is a typing mistake and should be an equals sign.
Unbelievable, how did I make such a mistake! Like you said, I changed intel_iommu_on to intel_iommu=on and it worked. Thank you so much!
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!