TASK ERROR: cannot prepare PCI pass-through, IOMMU not present (GPU passthrough)

[hamza_toor]

Maybe your Proxmox system does not use GRUB, as was the case for this person in this thread?

Alright! So, I'll check this.
I'll try creating cmdline file and adding the entry in that. However, just checked that I get the following output for this command before making the changes.
dmesg | grep -e DMAR -e IOMMU
[ 0.013228] ACPI: DMAR 0x000000006F7F0000 0001B8 (v01 DELL PE_SC3 00000001 DELL 00000001)
[ 0.013276] ACPI: Reserving DMAR table memory at [mem 0x6f7f0000-0x6f7f01b7]
[ 1.191593] DMAR: Host address width 46
[ 1.191595] DMAR: DRHD base: 0x000000d37fc000 flags: 0x0
[ 1.191603] DMAR: dmar0: reg_base_addr d37fc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[ 1.191608] DMAR: DRHD base: 0x000000e0ffc000 flags: 0x0
[ 1.191612] DMAR: dmar1: reg_base_addr e0ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[ 1.191616] DMAR: DRHD base: 0x000000ee7fc000 flags: 0x0
[ 1.191620] DMAR: dmar2: reg_base_addr ee7fc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[ 1.191623] DMAR: DRHD base: 0x000000fbffc000 flags: 0x0
[ 1.191628] DMAR: dmar3: reg_base_addr fbffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[ 1.191631] DMAR: DRHD base: 0x000000aaffc000 flags: 0x0
[ 1.191636] DMAR: dmar4: reg_base_addr aaffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[ 1.191638] DMAR: DRHD base: 0x000000b87fc000 flags: 0x0
[ 1.191643] DMAR: dmar5: reg_base_addr b87fc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[ 1.191646] DMAR: DRHD base: 0x000000c5ffc000 flags: 0x0
[ 1.191650] DMAR: dmar6: reg_base_addr c5ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[ 1.191653] DMAR: DRHD base: 0x0000009d7fc000 flags: 0x1
[ 1.191657] DMAR: dmar7: reg_base_addr 9d7fc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[ 1.191661] DMAR: RMRR base: 0x0000006e2c9000 end: 0x0000006e7c8fff
[ 1.191663] DMAR: RMRR base: 0x0000006f0c0000 end: 0x0000006f0c2fff
[ 1.191666] DMAR: [Firmware Bug]: No firmware reserved region can cover this RMRR [0x000000006f0c0000-0x000000006f0c2fff], contact BIOS vendor for fixes
[ 1.191669] DMAR: [Firmware Bug]: Your BIOS is broken; bad RMRR [0x000000006f0c0000-0x000000006f0c2fff]
[ 1.191674] DMAR: ATSR flags: 0x0
[ 1.191678] DMAR-IR: IOAPIC id 12 under DRHD base 0xc5ffc000 IOMMU 6
[ 1.191680] DMAR-IR: IOAPIC id 11 under DRHD base 0xb87fc000 IOMMU 5
[ 1.191682] DMAR-IR: IOAPIC id 10 under DRHD base 0xaaffc000 IOMMU 4
[ 1.191684] DMAR-IR: IOAPIC id 18 under DRHD base 0xfbffc000 IOMMU 3
[ 1.191686] DMAR-IR: IOAPIC id 17 under DRHD base 0xee7fc000 IOMMU 2
[ 1.191688] DMAR-IR: IOAPIC id 16 under DRHD base 0xe0ffc000 IOMMU 1
[ 1.191690] DMAR-IR: IOAPIC id 15 under DRHD base 0xd37fc000 IOMMU 0
[ 1.191692] DMAR-IR: IOAPIC id 8 under DRHD base 0x9d7fc000 IOMMU 7
[ 1.191694] DMAR-IR: IOAPIC id 9 under DRHD base 0x9d7fc000 IOMMU 7
[ 1.191696] DMAR-IR: HPET id 0 under DRHD base 0x9d7fc000
[ 1.191699] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit.
[ 1.191700] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the BIOS setting.
[ 1.193325] DMAR-IR: Enabled IRQ remapping in xapic mode


It says BIOS is broken. Is it normal? Can this be the reason for this issue?

 

[hamza_toor]

Maybe your Proxmox system does not use GRUB, as was the case for this person in this thread?

Alright! I'll check this

 

[hamza_toor]

Maybe your Proxmox system does not use GRUB, as was the case for this person in this thread?

@leesteken I did the same step as well. After editing the /etc/kernel/cmdline file and refreshing proxmox, I get the following output,

Running hook script 'proxmox-auto-removal'..
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
No /etc/kernel/proxmox-boot-uuids found, skipping ESP sync.

dmesg | grep -e DMAR -e IOMMU gives the following output

[    0.013228] ACPI: DMAR 0x000000006F7F0000 0001B8 (v01 DELL   PE_SC3   00000001 DELL 00000001)
[    0.013276] ACPI: Reserving DMAR table memory at [mem 0x6f7f0000-0x6f7f01b7]
[    1.191593] DMAR: Host address width 46
[    1.191595] DMAR: DRHD base: 0x000000d37fc000 flags: 0x0
[    1.191603] DMAR: dmar0: reg_base_addr d37fc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[    1.191608] DMAR: DRHD base: 0x000000e0ffc000 flags: 0x0
[    1.191612] DMAR: dmar1: reg_base_addr e0ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[    1.191616] DMAR: DRHD base: 0x000000ee7fc000 flags: 0x0
[    1.191620] DMAR: dmar2: reg_base_addr ee7fc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[    1.191623] DMAR: DRHD base: 0x000000fbffc000 flags: 0x0
[    1.191628] DMAR: dmar3: reg_base_addr fbffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[    1.191631] DMAR: DRHD base: 0x000000aaffc000 flags: 0x0
[    1.191636] DMAR: dmar4: reg_base_addr aaffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[    1.191638] DMAR: DRHD base: 0x000000b87fc000 flags: 0x0
[    1.191643] DMAR: dmar5: reg_base_addr b87fc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[    1.191646] DMAR: DRHD base: 0x000000c5ffc000 flags: 0x0
[    1.191650] DMAR: dmar6: reg_base_addr c5ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[    1.191653] DMAR: DRHD base: 0x0000009d7fc000 flags: 0x1
[    1.191657] DMAR: dmar7: reg_base_addr 9d7fc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[    1.191661] DMAR: RMRR base: 0x0000006e2c9000 end: 0x0000006e7c8fff
[    1.191663] DMAR: RMRR base: 0x0000006f0c0000 end: 0x0000006f0c2fff
[    1.191666] DMAR: [Firmware Bug]: No firmware reserved region can cover this RMRR [0x000000006f0c0000-0x000000006f0c2fff], contact BIOS vendor for fixes
[    1.191669] DMAR: [Firmware Bug]: Your BIOS is broken; bad RMRR [0x000000006f0c0000-0x000000006f0c2fff]
[    1.191674] DMAR: ATSR flags: 0x0
[    1.191678] DMAR-IR: IOAPIC id 12 under DRHD base  0xc5ffc000 IOMMU 6
[    1.191680] DMAR-IR: IOAPIC id 11 under DRHD base  0xb87fc000 IOMMU 5
[    1.191682] DMAR-IR: IOAPIC id 10 under DRHD base  0xaaffc000 IOMMU 4
[    1.191684] DMAR-IR: IOAPIC id 18 under DRHD base  0xfbffc000 IOMMU 3
[    1.191686] DMAR-IR: IOAPIC id 17 under DRHD base  0xee7fc000 IOMMU 2
[    1.191688] DMAR-IR: IOAPIC id 16 under DRHD base  0xe0ffc000 IOMMU 1
[    1.191690] DMAR-IR: IOAPIC id 15 under DRHD base  0xd37fc000 IOMMU 0
[    1.191692] DMAR-IR: IOAPIC id 8 under DRHD base  0x9d7fc000 IOMMU 7
[    1.191694] DMAR-IR: IOAPIC id 9 under DRHD base  0x9d7fc000 IOMMU 7
[    1.191696] DMAR-IR: HPET id 0 under DRHD base 0x9d7fc000
[    1.191699] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit.
[    1.191700] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the BIOS setting.
[    1.193325] DMAR-IR: Enabled IRQ remapping in xapic mode

 

[LeftShoe]

@leesteken I am having a similar issue to other users in this thread "TASK ERROR: cannot prepare PCI pass-through, IOMMU not present". I believe that my gpu is in the same group as my cpu and that is what is causing an issue, but I would like your input if possible.

My Specs:
Intel i5-10400
RTX A2000

VT-d is enabled in my BIOS

when running cat /proc/cmdline

I get this: BOOT_IMAGE=/boot/vmlinuz-5.15.108-1-pve root=/dev/mapper/pve-root ro single

The Entry for: /etc/default/grub

Is:
GRUB_DEFAULT=0 GRUB_TIMEOUT=5 GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian` GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction nofb nomodeset video=vesafb:off,efifb:off" GRUB_CMDLINE_LINUX=""

When running this: for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done

I get:
IOMMU group * 00:00.0 Host bridge [0600]: Intel Corporation Comet Lake-S 6c Host Bridge/DRAM Controller [8086:9b53] (rev 03) 00:01.0 PCI bridge [0604]: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 03) 00:04.0 Signal processing controller [1180]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Thermal Subsystem [8086:1903] (rev 03) 00:08.0 System peripheral [0880]: Intel Corporation Xeon E3-1200 v5/v6 / E3-1500 v5 / 6th/7th/8th Gen Core Processor Gaussian Mixture Model [8086:1911] 00:14.0 USB controller [0c03]: Intel Corporation Device [8086:43ed] (rev 11) 00:14.2 RAM memory [0500]: Intel Corporation Device [8086:43ef] (rev 11) 00:16.0 Communication controller [0780]: Intel Corporation Device [8086:43e0] (rev 11) 00:17.0 SATA controller [0106]: Intel Corporation Device [8086:43d2] (rev 11) 00:1c.0 PCI bridge [0604]: Intel Corporation Device [8086:43be] (rev 11) 00:1d.0 PCI bridge [0604]: Intel Corporation Device [8086:43b0] (rev 11) 00:1f.0 ISA bridge [0601]: Intel Corporation Device [8086:4387] (rev 11) 00:1f.3 Audio device [0403]: Intel Corporation Device [8086:f0c8] (rev 11) 00:1f.4 SMBus [0c05]: Intel Corporation Device [8086:43a3] (rev 11) 00:1f.5 Serial bus controller [0c80]: Intel Corporation Device [8086:43a4] (rev 11) 00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (11) I219-V [8086:0d4d] (rev 11) 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:2531] (rev a1) 01:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:228e] (rev a1) 02:00.0 SATA controller [0106]: ASMedia Technology Inc. Device [1b21:1064] (rev 02) 03:00.0 Non-Volatile memory controller [0108]: Sandisk Corp Device [15b7:5017] (rev 01)

When running: dmesg | grep -e DMAR -e IOMMU

I get this:
[ 0.010648] ACPI: DMAR 0x000000008E1FF000 000050 (v02 INTEL EDK2 00000002 01000013) [ 0.010700] ACPI: Reserving DMAR table memory at [mem 0x8e1ff000-0x8e1ff04f] [ 0.258659] DMAR: Host address width 39 [ 0.258661] DMAR: DRHD base: 0x000000fed91000 flags: 0x1 [ 0.258666] DMAR: dmar0: reg_base_addr fed91000 ver 1:0 cap d2008c40660462 ecap f050da [ 0.258672] DMAR-IR: IOAPIC id 2 under DRHD base 0xfed91000 IOMMU 0 [ 0.258675] DMAR-IR: HPET id 0 under DRHD base 0xfed91000 [ 0.258677] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping. [ 0.260111] DMAR-IR: Enabled IRQ remapping in x2apic mode

 

[leesteken]

@leesteken I am having a similar issue to other users in this thread "TASK ERROR: cannot prepare PCI pass-through, IOMMU not present". I believe that my gpu is in the same group as my cpu and that is what is causing an issue, but I would like your input if possible.

That's not the cause.

when running cat /proc/cmdline

I get this: BOOT_IMAGE=/boot/vmlinuz-5.15.108-1-pve root=/dev/mapper/pve-root ro single

intel_iommu=on is not there and therefore IOMMU is not enabled in the kernel (and everything is in group *).

Simply enable IOMMU by adding intel_iommu=on as the Proxmox manual describes (and ignore the remark that it is not necessary for kernel 5.15 and later). How to edit the kernel command line is described here in the manual. How to find out which bootloader is used is described here in the manual. All this is also in many other (recent) threads on this forum.

 

[Vikrant]

Thanks alot!
now it looks like it works

finaly editing the kernal commandline

nano /etc/kernel/cmdline ADD intel_iommu=on

than updating

proxmox-boot-tool refresh

dmesg | grep -e DMAR -e IOMMU
[ 0.011354] ACPI: DMAR 0x00000000A97EDAB0 0000DC (v01 DELL CBX3 00000001 INTL 20091013)
[ 0.011385] ACPI: Reserving DMAR table memory at [mem 0xa97edab0-0xa97edb8b]
[ 0.098610] DMAR: IOMMU enabled

VM also starting

I had the same issue and this solved my problem as well.
I was trying to pass a NIC card to my VM on a Dell R740.

Thank you.