Passthrough Intel 8970 to VM

[taylorjonl]

I am attempting to passthrough an Intel 8970 Quick Assist card to a VM so I can play around with it in VyOS. The host system is a Dell R730XD with a fresh install of 7.4(since the 8.x versions won't boot after a fresh install). I have been following this guide:

https://pve.proxmox.com/wiki/PCI(e)_Passthrough

Basically these are the devices I want to passthrough:

# lspci -nn | grep QuickAssist
84:00.0 Co-processor [0b40]: Intel Corporation C62x Chipset QuickAssist Technology [8086:37c8] (rev 04)
86:00.0 Co-processor [0b40]: Intel Corporation C62x Chipset QuickAssist Technology [8086:37c8] (rev 04)
88:00.0 Co-processor [0b40]: Intel Corporation C62x Chipset QuickAssist Technology [8086:37c8] (rev 04)

Through a lot of trial and error I finally have it so that vfio is taking ownership of the cards, here is some output from some common commands I found asked when people are having issues with passthrough:

# cat /proc/cmdline
initrd=\EFI\proxmox\5.15.102-1-pve\initrd.img-5.15.102-1-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on iommu=pt vfio_pci.disable_denylist=1

# dmesg | grep -e DMAR -e IOMMU -e AMD-Vi
[    0.011835] ACPI: DMAR 0x000000007B94B000 000130 (v01 DELL   PE_SC3   00000001 DELL 00000001)
[    0.011878] ACPI: Reserving DMAR table memory at [mem 0x7b94b000-0x7b94b12f]
[    1.536420] DMAR: IOMMU enabled
[    3.340079] DMAR: Host address width 46
[    3.340081] DMAR: DRHD base: 0x000000fbffc000 flags: 0x0
[    3.340089] DMAR: dmar0: reg_base_addr fbffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[    3.340093] DMAR: DRHD base: 0x000000c7ffc000 flags: 0x1
[    3.340099] DMAR: dmar1: reg_base_addr c7ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[    3.340103] DMAR: ATSR flags: 0x0
[    3.340105] DMAR: ATSR flags: 0x0
[    3.340110] DMAR-IR: IOAPIC id 10 under DRHD base  0xfbffc000 IOMMU 0
[    3.340113] DMAR-IR: IOAPIC id 8 under DRHD base  0xc7ffc000 IOMMU 1
[    3.340116] DMAR-IR: IOAPIC id 9 under DRHD base  0xc7ffc000 IOMMU 1
[    3.340119] DMAR-IR: HPET id 0 under DRHD base 0xc7ffc000
[    3.340122] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    3.340525] DMAR-IR: IRQ remapping was enabled on dmar0 but we are not in kdump mode
[    3.340925] DMAR-IR: IRQ remapping was enabled on dmar1 but we are not in kdump mode
[    3.341053] DMAR-IR: Enabled IRQ remapping in x2apic mode
[    4.139459] DMAR: No RMRR found
[    4.139461] DMAR: No SATC found
[    4.139465] DMAR: dmar0: Using Queued invalidation
[    4.139470] DMAR: dmar1: Using Queued invalidation
[    4.153563] DMAR: Intel(R) Virtualization Technology for Directed I/O

# lsmod | grep vfio
vfio_pci               16384  0
vfio_pci_core          73728  1 vfio_pci
vfio_virqfd            16384  1 vfio_pci_core
irqbypass              16384  2 vfio_pci_core,kvm
vfio_iommu_type1       45056  0
vfio                   45056  2 vfio_pci_core,vfio_iommu_type1

# lspci -k | grep -A 3 QuickAssist
84:00.0 Co-processor: Intel Corporation C62x Chipset QuickAssist Technology (rev 04)
        Subsystem: Intel Corporation QuickAssist Adapter 8970
        Kernel driver in use: vfio-pci
        Kernel modules: qat_c62x
86:00.0 Co-processor: Intel Corporation C62x Chipset QuickAssist Technology (rev 04)
        Subsystem: Intel Corporation QuickAssist Adapter 8970
        Kernel driver in use: vfio-pci
        Kernel modules: qat_c62x
88:00.0 Co-processor: Intel Corporation C62x Chipset QuickAssist Technology (rev 04)
        Subsystem: Intel Corporation QuickAssist Adapter 8970
        Kernel driver in use: vfio-pci
        Kernel modules: qat_c62x

The issue I am having is that these devices do not show up as available devices when trying to passthrough to the VM from the UI. Not sure what I am doing wrong but I am sorta at a point that I can't figure out how to proceed.