Can't passthrough my GPU to TrueNAS VM (Intel Arc A380 Low Profile)

[zenaror]

Hi!

I recently trying to migrate my homelab to proxmox VMs. The first VM that I tried to setup is my TrueNAS that run Plex and some small apps.. so I need passthrough my GPU, an Intel Arc A380 Low Profile.

But I'm receiving an error during when I start the VM

Running pre-start on VM=100
VM=100 - pre-start : Clearing Intel dGPU and audio device reset_methods.
Finished pre-start on VM=100
error writing '1' to '/sys/bus/pci/devices/0000:03:00.0/reset': Inappropriate ioctl for device
failed to reset PCI device '0000:03:00.0', but trying to continue as not all devices need a reset
error writing '1' to '/sys/bus/pci/devices/0000:04:00.0/reset': Inappropriate ioctl for device
failed to reset PCI device '0000:04:00.0', but trying to continue as not all devices need a reset
kvm: -device vfio-pci,host=0000:03:00.0,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0: vfio 0000:03:00.0: error getting device from group 15: No such device
Verify all devices in group 15 are bound to vfio-<bus> or pci-stub and not already in use
TASK ERROR: start failed: QEMU exited with code 1

This is my actual configs and specs:

• Motherboard Mancer H510M-DX (latest BIOS)
• Intel i5-11500
• 32GB RAM DDR4 (2x 16GB)
• 1x NVME SSD 240GB
• 2x 8TB HDD (WD NAS)
• 500w Power Supply
• AsRock Intel Arc A380 Low Profile

/etc/default/grub

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt pcie_aspm=off"

the pcie_aspm=off is to hide some PCIe messages:

pcieport 0000:00:01.0: AER: Correctable error message received from 0000:01:00.0
pcieport 0000:01:00.0: PCIe Bus Error: severity=Correctable, type=Physical Layer, (Receiver ID)
pcieport 0000:01:00.0: device [8086:4fa1] error status/mask=00000001/0000e000
pcieport 0000:01:00.0: [ 0] RxErr (First)

/etc/modules

# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.
# Parameters can be specified after the module name.

# Modules required for PCI passthrough
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

/etc/modprobe.d/blacklist.conf

blacklist i915
blacklist xe

/etc/modprobe.d/vfio.conf

#dGPU early alias
alias pci:v00008086d000056A5sv00001849sd00006006bc03sc00i00 vfio-pci

#dGPU Audio early alias
alias pci:v00008086d00004F92sv00001849sd00006006bc04sc03i00 vfio-pci

#dGPU + Audio
options vfio-pci ids=8086:56a5,8086:4f92 disable_vga=1

lspci -nnv (this "!!! Unknown header type 7f" message shows only after all configs)

03:00.0 VGA compatible controller [0300]: Intel Corporation DG2 [Arc A380] [8086:56a5] (rev 05) (prog-if 00 [VGA controller])
Subsystem: ASRock Incorporation DG2 [Arc A380] [1849:6006]
!!! Unknown header type 7f
Memory at a1000000 (64-bit, non-prefetchable) [size=16M]
Memory at 6000000000 (64-bit, prefetchable) [size=256M]
Expansion ROM at a2000000 [disabled] [size=2M]
Kernel driver in use: vfio-pci
Kernel modules: i915, xe

04:00.0 Audio device [0403]: Intel Corporation DG2 Audio Controller [8086:4f92]
Subsystem: ASRock Incorporation DG2 Audio Controller [1849:6006]
!!! Unknown header type 7f
Memory at a2300000 (64-bit, non-prefetchable) [size=16K]
Kernel driver in use: vfio-pci
Kernel modules: snd_hda_intel

but this was the output before:

03:00.0 VGA compatible controller [0300]: Intel Corporation DG2 [Arc A380] [8086:56a5] (rev 05) (prog-if 00 [VGA controller])
Subsystem: ASRock Incorporation DG2 [Arc A380] [1849:6006]
Flags: bus master, fast devsel, latency 0, IOMMU group 15
Memory at a1000000 (64-bit, non-prefetchable) [size=16M]
Memory at 6000000000 (64-bit, prefetchable) [size=8G]
Expansion ROM at a2000000 [disabled] [size=2M]
Capabilities: [40] Vendor Specific Information: Len=0c <?>
Capabilities: [70] Express Endpoint, MSI 00
Capabilities: [ac] MSI: Enable- Count=1/1 Maskable+ 64bit+
Capabilities: [d0] Power Management version 3
Capabilities: [100] Alternative Routing-ID Interpretation (ARI)
Capabilities: [420] Physical Resizable BAR
Capabilities: [400] Latency Tolerance Reporting
Kernel driver in use: vfio-pci
Kernel modules: i915, xe

04:00.0 Audio device [0403]: Intel Corporation DG2 Audio Controller [8086:4f92]
Subsystem: ASRock Incorporation DG2 Audio Controller [1849:6006]
Flags: fast devsel, IOMMU group 16
Memory at a2300000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [50] Power Management version 3
Capabilities: [60] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [70] Express Endpoint, MSI 00
Capabilities: [100] Latency Tolerance Reporting
Kernel modules: snd_hda_intel

100.conf

bios: ovmf
boot: order=net0
cores: 2
cpu: host
efidisk0: local-lvm:vm-100-disk-0,efitype=4m,pre-enrolled-keys=1,size=4M
hookscript: local:snippets/intel-dGPU-hookscript.sh
hostpci0: 0000:03:00,pcie=1
hostpci1: 0000:04:00,pcie=1
machine: q35
memory: 8192
meta: creation-qemu=9.2.0,ctime=1746636162
name: TrueNAS
net0: virtio=BC:24:11:24:6C:96,bridge=vmbr0,firewall=1
numa: 0
ostype: l26
scsihw: virtio-scsi-single
smbios1: uuid=ccf8f9a0-3e7d-44bd-b64f-b44325657958
sockets: 1
vmgenid: dfe99c6e-5a7c-46d5-ad2d-0c06b7ab1fb6

intel-dGPU-hookscript.sh (an attempt to fix the "Unable to change power state from D3cold to D0,device inaccessible" error)

#!/bin/bash
set -e -o errexit -o pipefail -o nounset

# located in /var/lib/vz/snippets/intel-dGPU-hookscript.sh or your 'snippets' location.
# Add to VM via: qm set VMID --hookscript local:snippets/intel-dGPU-hookscript.sh

# Do not modify these variables (set by Proxmox when calling the script)
vmId="$1"
runPhase="$2"
echo "Running $runPhase on VM=$vmId"

case "$runPhase" in
pre-start)
# Clear the reset methods before each start of the VM, to prevent the PVE host locking up
# flr and bus methods dont work and re-appear after reboots.
# bus method may still be attempted in subseqent VM starts, even if reset_method already cleared.
echo "VM=$vmId - $runPhase : Clearing Intel dGPU and audio device reset_methods."
echo > /sys/bus/pci/devices/0000:03:00.0/reset_method
echo > /sys/bus/pci/devices/0000:04:00.0/reset_method

# will appear as the following in journalctl:
# kernel: vfio-pci 0000:03:00.0: All device reset methods disabled by user
# kernel: vfio-pci 0000:04:00.0: All device reset methods disabled by user
;;

post-start)
# placeholder .
echo "VM=$vmId - $runPhase : No Action."
;;

pre-stop)
# placeholder .
echo "VM=$vmId - $runPhase : No Action."
;;
post-stop)
# placeholder .
echo "VM=$vmId - $runPhase : No Action."
;;
*)
echo "Unknown run phase \"$runPhase\"!"
;;
esac
echo "Finished $runPhase on VM=$vmId"

I also disabled the resizable bar and 4g decoding