[SOLVED] PCI Passthrough issue

dabaers

Member
Oct 9, 2021
38
0
6
28
Hey guys, second run at trying to get PCI passthrough working within proxmox,

Server - 2630L v4, supermicro x10srl-f, Quadro T400 and Intel 4x1gbe nic

GRUB
If you change this file, run 'update-grub' afterwards to update
# /boot/grub/grub.cfg.
# For full documentation of the options in this file, see:
# info -f grub -n 'Simple configuration'

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
#GRUD_DISTRIBUTOR="Proxmox Virtual Environment"
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
#GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on pcie_acs_override=downstream,multifunction v>
#GRUB_CMDLINE_LINUX_DEFAULT="quiet"
GRUB_CMDLINE_LINUX=""

/etc/kernal/cmdline
root=ZFS=rpool/ROOT/pve-1 boot=zfs quiet intel_iommu=on

DMAR
[ 0.014584] ACPI: DMAR 0x0000000079863C50 0000BC (v01 SUPERM SMCI--MB 00000001 INTL 20091013)
[ 0.014641] ACPI: Reserving DMAR table memory at [mem 0x79863c50-0x79863d0b]
[ 0.395982] DMAR: Host address width 46
[ 0.395985] DMAR: DRHD base: 0x000000fbffc000 flags: 0x1
[ 0.395993] DMAR: dmar0: reg_base_addr fbffc000 ver 1:0 cap 8d2078c106f0466 ecap f020de
[ 0.395998] DMAR: RMRR base: 0x0000007ba18000 end: 0x0000007ba27fff
[ 0.396002] DMAR: ATSR flags: 0x0
[ 0.396005] DMAR: RHSA base: 0x000000fbffc000 proximity domain: 0x0
[ 0.396010] DMAR-IR: IOAPIC id 1 under DRHD base 0xfbffc000 IOMMU 0
[ 0.396013] DMAR-IR: IOAPIC id 2 under DRHD base 0xfbffc000 IOMMU 0
[ 0.396017] DMAR-IR: HPET id 0 under DRHD base 0xfbffc000
[ 0.396020] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[ 0.396326] DMAR-IR: Enabled IRQ remapping in x2apic mode

find /sys/kernel/iommu_groups/ -type l
(no return)

Bios level
Virtualization, SRIOV and X2apic have been enabled also above 4g decoding and efi oprom for both pci ports have been selected.

Error in Vm is states that NO IOMMU DETECTED; additionally, this board is stated to have C612 chipset (which is SR-IOV and VT-d) but proxmox reads c610 chipset in PCI selection menu.

I need help with this if anything can!
 

leesteken

Famous Member
May 31, 2020
2,384
509
118
Have a look at cat /proc/cmdline to double check that intel_iommu=on is actually active.
If find /sys/kernel/iommu_groups/ -type l is empty then IOMMU/VT-d is not (fully) enabled (in the BIOS and kernel) or not supported (by the hardware or the kernel).
 

dabaers

Member
Oct 9, 2021
38
0
6
28
I would be really surprised if vt-d/iommu didn't work on this board and CPU as advertised, and it is enterprise gear.

cat /proc/cmdline

initrd=\EFI\proxmox\5.13.19-2-pve\initrd.img-5.13.19-2-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs
 

leesteken

Famous Member
May 31, 2020
2,384
509
118
initrd=\EFI\proxmox\5.13.19-2-pve\initrd.img-5.13.19-2-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs

As you can see, there is no intel_iommu=on in that line, therefore it is not enabled (in the kernel). It looks like you changed GRUB but Proxmox uses systemd-boot for your system.
Please add intel_iommu=on at the end of the first line in /etc/kernel/cmdline and run proxmox-boot-tool refresh before rebooting.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!