[SOLVED] PCI Passthrough Issue on Dell Precision 7920 with Proxmox

[Himala]

Hello everyone,

I'm in the process of setting up PCI passthrough on a Dell Precision 7920 workstation running Proxmox. In the Dell BIOS, I've enabled all available settings for IOMMU and PCI passthrough. However, I'm still facing the "No IOMMU detected, please activate it" error despite having followed all the steps meticulously from the Proxmox wiki.

Has anyone had success with PCI passthrough on this specific Dell model? Any pointers or advice would be greatly appreciated.

For some context, I previously had VMWARE ESXi installed on this workstation, and PCI passthrough worked flawlessly. This makes me confident that the BIOS settings are correct.
System Configuration:

• Model: Dell Precision 7920
• CPUs: Dual Intel XEON Gold 6138
• RAM: 512GB

When I execute `dmesg | grep -e DMAR -e IOMMU`, I receive the following output:

root@proxmox01:~# dmesg | grep -e DMAR -e IOMMU
[    0.020429] ACPI: DMAR 0x0000000069648768 000228 (v01 DELL\x CBX3     00000001 INTL 20091013)
[    0.020496] ACPI: Reserving DMAR table memory at [mem 0x69648768-0x6964898f]
[    2.638716] DMAR: Host address width 46
[    2.638720] DMAR: DRHD base: 0x000000c63fc000 flags: 0x0
[    2.638731] DMAR: dmar0: reg_base_addr c63fc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[    2.638738] DMAR: DRHD base: 0x000000cfbfc000 flags: 0x0
[    2.638746] DMAR: dmar1: reg_base_addr cfbfc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[    2.638752] DMAR: DRHD base: 0x000000d37fc000 flags: 0x0
[    2.638759] DMAR: dmar2: reg_base_addr d37fc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[    2.638764] DMAR: DRHD base: 0x000000fbffc000 flags: 0x0
[    2.638771] DMAR: dmar3: reg_base_addr fbffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[    2.638776] DMAR: DRHD base: 0x000000b33fc000 flags: 0x0
[    2.638786] DMAR: dmar4: reg_base_addr b33fc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[    2.638791] DMAR: DRHD base: 0x000000b63fc000 flags: 0x0
[    2.638798] DMAR: dmar5: reg_base_addr b63fc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[    2.638803] DMAR: DRHD base: 0x000000c5ffc000 flags: 0x0
[    2.638809] DMAR: dmar6: reg_base_addr c5ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[    2.638814] DMAR: DRHD base: 0x000000903fc000 flags: 0x1
[    2.638820] DMAR: dmar7: reg_base_addr 903fc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[    2.638826] DMAR: RMRR base: 0x0000006baab000 end: 0x0000006babbfff
[    2.638831] DMAR: ATSR flags: 0x0
[    2.638834] DMAR: RHSA base: 0x000000903fc000 proximity domain: 0x0
[    2.638839] DMAR: RHSA base: 0x000000b33fc000 proximity domain: 0x0
[    2.638842] DMAR: RHSA base: 0x000000b63fc000 proximity domain: 0x0
[    2.638846] DMAR: RHSA base: 0x000000c5ffc000 proximity domain: 0x0
[    2.638850] DMAR: RHSA base: 0x000000c63fc000 proximity domain: 0x1
[    2.638853] DMAR: RHSA base: 0x000000cfbfc000 proximity domain: 0x1
[    2.638856] DMAR: RHSA base: 0x000000d37fc000 proximity domain: 0x1
[    2.638860] DMAR: RHSA base: 0x000000fbffc000 proximity domain: 0x1
[    2.638865] DMAR-IR: IOAPIC id 12 under DRHD base  0xc5ffc000 IOMMU 6
[    2.638870] DMAR-IR: IOAPIC id 11 under DRHD base  0xb63fc000 IOMMU 5
[    2.638874] DMAR-IR: IOAPIC id 10 under DRHD base  0xb33fc000 IOMMU 4
[    2.638878] DMAR-IR: IOAPIC id 18 under DRHD base  0xfbffc000 IOMMU 3
[    2.638882] DMAR-IR: IOAPIC id 17 under DRHD base  0xd37fc000 IOMMU 2
[    2.638885] DMAR-IR: IOAPIC id 16 under DRHD base  0xcfbfc000 IOMMU 1
[    2.638889] DMAR-IR: IOAPIC id 15 under DRHD base  0xc63fc000 IOMMU 0
[    2.638893] DMAR-IR: IOAPIC id 8 under DRHD base  0x903fc000 IOMMU 7
[    2.638897] DMAR-IR: IOAPIC id 9 under DRHD base  0x903fc000 IOMMU 7
[    2.638901] DMAR-IR: HPET id 0 under DRHD base 0x903fc000
[    2.638905] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    2.641530] DMAR-IR: Enabled IRQ remapping in x2apic mode

Additionally, here's my current GRUB configuration:

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction nofb nomodeset video=vesafb:off,efifb:off"
#GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"
#GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on"
GRUB_CMDLINE_LINUX=""

Passthrough Intentions:

I'm aiming to passthrough the following devices to a Windows VM:

• PCI SATA Controller
• Network Card
• Possibly a GPU in the future
  

I would genuinely appreciate any guidance or suggestions you can provide. I really appreciate any help you can provide.

 

[leesteken]

Does your CPU and motherboard support Intel VT-d? I don't know which Dell computer supports it and which ones don't.
Maybe your Proxmox does not use GRUB? See the manual on how to check.
What is the output of cat /proc/cmdline (which shows the currently active parameters)?

Also note that video=vesafb:off,efifb:off is invalid and (even corrected) won't do anything on Proxmox.

 

[Himala]

Does your CPU and motherboard support Intel VT-d? I don't know which Dell computer supports it and which ones don't.
Maybe your Proxmox does not use GRUB? See the manual on how to check.
What is the output of cat /proc/cmdline (which shows the currently active parameters)?

Also note that video=vesafb:off,efifb:off is invalid and (even corrected) won't do anything on Proxmox.

root@proxmox01:~# cat /proc/cmdline
initrd=\EFI\proxmox\6.2.16-3-pve\initrd.img-6.2.16-3-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs
root@proxmox01:~

Yes VT-d si enable as i mention With ESXi works)

 

[leesteken]

initrd=\EFI\proxmox\6.2.16-3-pve\initrd.img-6.2.16-3-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs

Oh look, there is no intel_iommu=on there! Please follow the manual in adding and activating this: https://pve.proxmox.com/pve-docs/pve-admin-guide.html#sysboot_edit_kernel_cmdline. Hint: follow the systemd-boot steps.

 

[Himala]

I got it working. I got the solution via Evan Miller on the Facebook group

This is What Evan Miller Ask me to do.

nano /etc/default/grub
Then change this line to:
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"
update-grub
nano /etc/modules
Then paste
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
Then
echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf
echo "options kvm ignore_msrs=1" > /etc/modprobe.d/kvm.conf
nano /etc/kernel/cmdline
root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on

Then run
proxmox-boot-tool refresh

 

[leesteken]

I got it working. I got the solution via Evan Miller on the Facebook group

This is What Evan Miller Ask me to do.

nano /etc/default/grub
Then change this line to:
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"
update-grub
nano /etc/modules
Then paste
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
Then
echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf
echo "options kvm ignore_msrs=1" > /etc/modprobe.d/kvm.conf
nano /etc/kernel/cmdline
root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on

Then run
proxmox-boot-tool refresh

vfio_virqfd is no longer a module, and all you had to do was follow the manual.

PS: Let me know next time you are racing several support groups against each other...