pci device exists in vfio-pci device denylist, driver probing disallowed.

[Impovich]

Hello all,
trying to passthrough QAT adapter to a VM which is not listed in UI, so added hostpci0: 54:00.0;55:00.0;56:00.0 to vmid.conf but VM won't start throwing an error

May  6 22:43:18 dallas pvedaemon[29431]: Cannot bind 0000:54:00.0 to vfio
May  6 22:43:18 dallas kernel: [  954.660069] vfio-pci 0000:54:00.0: 8086:37c8 exists in vfio-pci device denylist, driver probing disallowed.
May  6 22:43:18 dallas kernel: [  954.660079] vfio-pci: probe of 0000:54:00.0 failed with error -22

lspci:

54:00.0 Co-processor: Intel Corporation C62x Chipset QuickAssist Technology (rev 04)
55:00.0 Co-processor: Intel Corporation C62x Chipset QuickAssist Technology (rev 04)
56:00.0 Co-processor: Intel Corporation C62x Chipset QuickAssist Technology (rev 04)

root@dallas:~# dmesg | grep -e DMAR -e IOMMU
[    0.027377] ACPI: DMAR 0x000000005F164000 00017C (v01 SUPERM SMCI--MB 00000001 INTL 20091013)
[    0.027429] ACPI: Reserving DMAR table memory at [mem 0x5f164000-0x5f16417b]
[    0.129898] DMAR: IOMMU enabled
[    0.323561] DMAR: Host address width 46
[    0.323562] DMAR: DRHD base: 0x000000bb7fc000 flags: 0x0
[    0.323568] DMAR: dmar0: reg_base_addr bb7fc000 ver 4:0 cap 8ed008c40780466 ecap 60000f050df
[    0.323572] DMAR: DRHD base: 0x000000d0ffc000 flags: 0x0
[    0.323577] DMAR: dmar1: reg_base_addr d0ffc000 ver 4:0 cap 8ed008c40780466 ecap 60000f050df
[    0.323579] DMAR: DRHD base: 0x000000e67fc000 flags: 0x0
[    0.323582] DMAR: dmar2: reg_base_addr e67fc000 ver 4:0 cap 8ed008c40780466 ecap 60000f050df
[    0.323584] DMAR: DRHD base: 0x000000fb7fc000 flags: 0x0
[    0.323587] DMAR: dmar3: reg_base_addr fb7fc000 ver 4:0 cap 8ed008c40780466 ecap 60000f050df
[    0.323589] DMAR: DRHD base: 0x000000a5ffc000 flags: 0x1
[    0.323592] DMAR: dmar4: reg_base_addr a5ffc000 ver 4:0 cap 8ed008c40780466 ecap 60000f050df
[    0.323595] DMAR: RMRR base: 0x0000006b985000 end: 0x0000006b9a8fff
[    0.323597] DMAR: RMRR base: 0x0000006a3dd000 end: 0x0000006a626fff
[    0.323598] DMAR: ATSR flags: 0x0
[    0.323600] DMAR: RHSA base: 0x000000a5ffc000 proximity domain: 0x0
[    0.323601] DMAR: RHSA base: 0x000000bb7fc000 proximity domain: 0x0
[    0.323602] DMAR: RHSA base: 0x000000d0ffc000 proximity domain: 0x0
[    0.323603] DMAR: RHSA base: 0x000000e67fc000 proximity domain: 0x0
[    0.323604] DMAR: RHSA base: 0x000000fb7fc000 proximity domain: 0x0
[    0.323606] DMAR-IR: IOAPIC id 8 under DRHD base  0xa5ffc000 IOMMU 4
[    0.323608] DMAR-IR: HPET id 0 under DRHD base 0xa5ffc000
[    0.323609] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.325325] DMAR-IR: Enabled IRQ remapping in x2apic mode
[    1.510178] DMAR: No SATC found
[    1.510181] DMAR: dmar2: Using Queued invalidation
[    1.510185] DMAR: dmar1: Using Queued invalidation
[    1.510197] DMAR: dmar0: Using Queued invalidation
[    1.510199] DMAR: dmar4: Using Queued invalidation
[    1.518198] DMAR: Intel(R) Virtualization Technology for Directed I/O

root@dallas:~# cat /etc/modules
# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

root@dallas:/# cat /proc/cmdline
initrd=\EFI\proxmox\5.15.102-1-pve\initrd.img-5.15.102-1-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs quiet intel_iommu=on iommu=pt

 

[Impovich]

Looks like i found an answer to my question:
https://01.org/sites/default/files/downloads/330689intelqatvirtualizationappnoterev0110.pdf

Note: On more recent kernels, it may be required to have the vfio-pci module inserted with
“disable_denylist=1”. Note that this can be done once with “sudo modprobe vfio-pci
disable_denylist=1” or persistently by adding the option “options vfio-pci
disable_denylist=1” to /etc/modprobe.d/vfio-pci.conf.