PCI Passthrough Arc A380 Into VM

T

trademarkej

New Member
Mar 6, 2024
6
0
1
I recently setup a new instance of Proxmox 8.2.2 after my Intel i5 10500 died on an AMD 5700UG based system and haven't had much luck getting iGPU transcoding working (so much easier on my old Core i5) so I opted to install an ASROCK Intel Arc A380 to do hardware transcoding in a kubernetes pod running on an Ubuntu 23.10 cloud image VM. I believe I have everything configured correctly and can add the A380 devices into the test VM I spun up as PCI devices without any errors but I can't seem to get drivers to load (weren't they supposed to be native in ~> 6.5 kernels). Any help on how to proceed with getting this card working would be greatly appreciated!

/boot/grub/grub.cfg
Bash: 
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"

/etc/modules
Bash: 
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

/etc/modprobe.d/blacklist.conf
Bash: 
blacklist i915

/etc/modprobe.d/vfio.conf
Bash: 
options vfio-pci ids=8086:56a5,8086:4f92 disable_vga=1

100.conf
Bash: 
agent: enabled=1
balloon: 0
boot: c
bootdisk: scsi0
cores: 2
hostpci0: 0000:09:00,pcie=1
hostpci1: 0000:0a:00,pcie=1
ide2: local-lvm:vm-100-cloudinit,media=cdrom,size=4M
machine: q35
memory: 4096
name: test-pci
ostype: l26
scsi0: local-lvm:vm-100-disk-0,discard=on,size=10752M,ssd=1
scsihw: virtio-scsi-pci
serial0: socket
vga: serial0

Bash: 
$ lspci -nnv
09:00.0 VGA compatible controller [0300]: Intel Corporation DG2 [Arc A380] [8086:56a5] (rev 05) (prog-if 00 [VGA controller])
        Subsystem: ASRock Incorporation DG2 [Arc A380] [1849:6004]
        Flags: fast devsel, IOMMU group 19
        Memory at fb000000 (64-bit, non-prefetchable) [size=16M]
        Memory at b0000000 (64-bit, prefetchable) [size=256M]
        Expansion ROM at fc000000 [disabled] [size=2M]
        Capabilities: [40] Vendor Specific Information: Len=0c <?>
        Capabilities: [70] Express Endpoint, MSI 00
        Capabilities: [ac] MSI: Enable- Count=1/1 Maskable+ 64bit+
        Capabilities: [d0] Power Management version 3
        Capabilities: [100] Alternative Routing-ID Interpretation (ARI)
        Capabilities: [420] Physical Resizable BAR
        Capabilities: [400] Latency Tolerance Reporting
        Kernel driver in use: vfio-pci
        Kernel modules: i915, xe

0a:00.0 Audio device [0403]: Intel Corporation DG2 Audio Controller [8086:4f92]
        Subsystem: ASRock Incorporation DG2 Audio Controller [1849:6004]
        Flags: fast devsel, IOMMU group 20
        Memory at fc300000 (64-bit, non-prefetchable) [size=16K]
        Capabilities: [50] Power Management version 3
        Capabilities: [60] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [70] Express Endpoint, MSI 00
        Capabilities: [100] Latency Tolerance Reporting
        Kernel driver in use: vfio-pci
        Kernel modules: snd_hda_intel

$ lspci -n -s 09:00
09:00.0 0300: 8086:56a5 (rev 05)
$ lspci -n -s 0a:00
0a:00.0 0403: 8086:4f92
 
I /just/ got the a380 passed through to my win10 vm today, so this pain is fresh for me. I can't solve your exact problem but I can provide my settings for you to compare.

cat /etc/default/grub (my proxmox install is using legacy bios)
Bash: 
GRUB_CMDLINE_LINUX_DEFAULT="quiet and intel_iommu=on iommu=pt"

cat /etc/modules
Bash: 
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

cat /etc/modprobe.d/blacklist.conf
Bash: 
blacklist i915
blacklist xe

cat /etc/modprobe.d/vfio.conf - I got the alias bit from here: https://null-src.com/posts/qemu-vfio-pci/post.php - is it needed? unknown but I have been throwing everything I can find to a solution.
Bash: 
alias pci:v00008086d000056A5sv00001849sd00006004bc03sc00i00 vfio-pci
alias pci:v00008086d00004F92sv00001849sd00006004bc04sc03i00 vfio-pci
options vfio-pci ids=8086:56a5,8086:4f92 disable_vga=1

lspci -nnv (notice the two kernel modules - xe and i915 this is why I put them both in the blacklist - needed ? I really dont know just content with my win for the day.
Bash: 
05:00.0 VGA compatible controller [0300]: Intel Corporation DG2 [Arc A380] [8086:56a5] (rev 05) (prog-if 00 [VGA controller])
        Subsystem: ASRock Incorporation DG2 [Arc A380] [1849:6004]
        Flags: bus master, fast devsel, latency 0, IRQ 249, NUMA node 0, IOMMU group 53
        Memory at 94000000 (64-bit, non-prefetchable) [size=16M]
        Memory at 3bfe0000000 (64-bit, prefetchable) [size=256M]
        Expansion ROM at <ignored> [disabled]
        Capabilities: [40] Vendor Specific Information: Len=0c <?>
        Capabilities: [70] Express Endpoint, MSI 00
        Capabilities: [ac] MSI: Enable+ Count=1/1 Maskable+ 64bit+
        Capabilities: [d0] Power Management version 3
        Capabilities: [100] Alternative Routing-ID Interpretation (ARI)
        Capabilities: [420] Physical Resizable BAR
        Capabilities: [400] Latency Tolerance Reporting
        Kernel driver in use: vfio-pci
        Kernel modules: i915, xe

06:00.0 Audio device [0403]: Intel Corporation DG2 Audio Controller [8086:4f92]
        Subsystem: ASRock Incorporation DG2 Audio Controller [1849:6004]
        Flags: bus master, fast devsel, latency 0, IRQ 250, NUMA node 0, IOMMU group 54
        Memory at 95000000 (64-bit, non-prefetchable) [size=16K]
        Capabilities: [50] Power Management version 3
        Capabilities: [60] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Capabilities: [70] Express Endpoint, MSI 00
        Capabilities: [100] Latency Tolerance Reporting
        Kernel driver in use: vfio-pci
        Kernel modules: snd_hda_intel

My VM config to assist someone in the future or me in case I forget to back it up before doing something stupid.
Bash: 
agent: 1
bios: ovmf
boot: order=scsi0;ide0;net0
cores: 12
cpu: host
efidisk0: local-lvm:vm-115-disk-1,efitype=4m,pre-enrolled-keys=1,size=4M
--> DG2 Video --> hostpci0: 0000:05:00,pcie=1
--> DG2 Audio --> hostpci1: 0000:06:00,pcie=1
ide0: NFS-ISO:iso/virtio-win-0.1.240.iso,media=cdrom,size=612812K
machine: pc-q35-8.1
memory: 32000
meta: creation-qemu=8.1.5,ctime=1715857038
name: BI
net0: virtio=BC:24:11:BE:1C:51,bridge=vmbr1,firewall=1,mtu=9000,tag=20
numa: 0
ostype: win11
scsi0: local-lvm:vm-115-disk-0,cache=writeback,discard=on,iothread=1,size=1000G
scsihw: virtio-scsi-single
smbios1: uuid=(id here)
sockets: 2
vga: none
vmgenid: (id here)
 
Last edited:
  • Like
Reactions: trademarkej
SoggyCrouton said:
I /just/ got the a380 passed through to my win10 vm today, so this pain is fresh for me. I can't solve your exact problem but I can provide my settings for you to compare.

cat /etc/default/grub (my proxmox install is using legacy bios)
Bash: 
GRUB_CMDLINE_LINUX_DEFAULT="quiet and intel_iommu=on iommu=pt"

cat /etc/modules
Bash: 
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

cat /etc/modprobe.d/blacklist.conf
Bash: 
blacklist i915
blacklist xe

cat /etc/modprobe.d/vfio.conf - I got the alias bit from here: https://null-src.com/posts/qemu-vfio-pci/post.php - is it needed? unknown but I have been throwing everything I can find to a solution.
Bash: 
alias pci:v00008086d000056A5sv00001849sd00006004bc03sc00i00 vfio-pci
alias pci:v00008086d00004F92sv00001849sd00006004bc04sc03i00 vfio-pci
options vfio-pci ids=8086:56a5,8086:4f92 disable_vga=1

lspci -nnv (notice the two kernel modules - xe and i915 this is why I put them both in the blacklist - needed ? I really dont know just content with my win for the day.
Bash: 
05:00.0 VGA compatible controller [0300]: Intel Corporation DG2 [Arc A380] [8086:56a5] (rev 05) (prog-if 00 [VGA controller])
        Subsystem: ASRock Incorporation DG2 [Arc A380] [1849:6004]
        Flags: bus master, fast devsel, latency 0, IRQ 249, NUMA node 0, IOMMU group 53
        Memory at 94000000 (64-bit, non-prefetchable) [size=16M]
        Memory at 3bfe0000000 (64-bit, prefetchable) [size=256M]
        Expansion ROM at <ignored> [disabled]
        Capabilities: [40] Vendor Specific Information: Len=0c <?>
        Capabilities: [70] Express Endpoint, MSI 00
        Capabilities: [ac] MSI: Enable+ Count=1/1 Maskable+ 64bit+
        Capabilities: [d0] Power Management version 3
        Capabilities: [100] Alternative Routing-ID Interpretation (ARI)
        Capabilities: [420] Physical Resizable BAR
        Capabilities: [400] Latency Tolerance Reporting
        Kernel driver in use: vfio-pci
        Kernel modules: i915, xe

06:00.0 Audio device [0403]: Intel Corporation DG2 Audio Controller [8086:4f92]
        Subsystem: ASRock Incorporation DG2 Audio Controller [1849:6004]
        Flags: bus master, fast devsel, latency 0, IRQ 250, NUMA node 0, IOMMU group 54
        Memory at 95000000 (64-bit, non-prefetchable) [size=16K]
        Capabilities: [50] Power Management version 3
        Capabilities: [60] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Capabilities: [70] Express Endpoint, MSI 00
        Capabilities: [100] Latency Tolerance Reporting
        Kernel driver in use: vfio-pci
        Kernel modules: snd_hda_intel

My VM config to assist someone in the future or me in case I forget to back it up before doing something stupid.
Bash: 
agent: 1
bios: ovmf
boot: order=scsi0;ide0;net0
cores: 12
cpu: host
efidisk0: local-lvm:vm-115-disk-1,efitype=4m,pre-enrolled-keys=1,size=4M
--> DG2 Video --> hostpci0: 0000:05:00,pcie=1
--> DG2 Audio --> hostpci1: 0000:06:00,pcie=1
ide0: NFS-ISO:iso/virtio-win-0.1.240.iso,media=cdrom,size=612812K
machine: pc-q35-8.1
memory: 32000
meta: creation-qemu=8.1.5,ctime=1715857038
name: BI
net0: virtio=BC:24:11:BE:1C:51,bridge=vmbr1,firewall=1,mtu=9000,tag=20
numa: 0
ostype: win11
scsi0: local-lvm:vm-115-disk-0,cache=writeback,discard=on,iothread=1,size=1000G
scsihw: virtio-scsi-single
smbios1: uuid=(id here)
sockets: 2
vga: none
vmgenid: (id here)
Click to expand...

Ok, so I checked my VM using lscpi -nnv and it appears that both the video and audio devices are listed. I don't see any listing for the drivers being loaded though.

Bash: 
$ uname -srm
Linux 6.8.0-31-generic x86_64

$ lscpi -nnv
01:00.0 VGA compatible controller [0300]: Intel Corporation DG2 [Arc A380] [8086:56a5] (rev 05) (prog-if 00 [VGA controller])
        Subsystem: ASRock Incorporation DG2 [Arc A380] [1849:6004]
        Physical Slot: 0
        Flags: fast devsel
        Memory at fb000000 (64-bit, non-prefetchable) [size=16M]
        Memory at e0000000 (64-bit, prefetchable) [size=256M]
        Expansion ROM at fc000000 [disabled] [size=2M]
        Capabilities: [40] Vendor Specific Information: Len=0c <?>
        Capabilities: [70] Express Endpoint, MSI 00
        Capabilities: [ac] MSI: Enable- Count=1/1 Maskable+ 64bit+
        Capabilities: [d0] Power Management version 3
        Capabilities: [100] Null
        Capabilities: [420] Physical Resizable BAR
        Capabilities: [400] Latency Tolerance Reporting

02:00.0 Audio device [0403]: Intel Corporation DG2 Audio Controller [8086:4f92]
        Subsystem: ASRock Incorporation DG2 Audio Controller [1849:6004]
        Physical Slot: 0-2
        Flags: fast devsel
        Memory at fde00000 (64-bit, non-prefetchable) [size=16K]
        Capabilities: [50] Power Management version 3
        Capabilities: [60] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [70] Express Endpoint, MSI 00
        Capabilities: [100] Latency Tolerance Reporting

What am I still missing?
 
Last edited:
trademarkej said:
Ok, so I checked my VM using lscpi -nnv and it appears that both the video and audio devices are listed. I don't see any listing for the drivers being loaded though.

Bash: 
$ uname -srm
Linux 6.8.0-31-generic x86_64

$ lscpi -nnv
01:00.0 VGA compatible controller [0300]: Intel Corporation DG2 [Arc A380] [8086:56a5] (rev 05) (prog-if 00 [VGA controller])
        Subsystem: ASRock Incorporation DG2 [Arc A380] [1849:6004]
        Physical Slot: 0
        Flags: fast devsel
        Memory at fb000000 (64-bit, non-prefetchable) [size=16M]
        Memory at e0000000 (64-bit, prefetchable) [size=256M]
        Expansion ROM at fc000000 [disabled] [size=2M]
        Capabilities: [40] Vendor Specific Information: Len=0c <?>
        Capabilities: [70] Express Endpoint, MSI 00
        Capabilities: [ac] MSI: Enable- Count=1/1 Maskable+ 64bit+
        Capabilities: [d0] Power Management version 3
        Capabilities: [100] Null
        Capabilities: [420] Physical Resizable BAR
        Capabilities: [400] Latency Tolerance Reporting

02:00.0 Audio device [0403]: Intel Corporation DG2 Audio Controller [8086:4f92]
        Subsystem: ASRock Incorporation DG2 Audio Controller [1849:6004]
        Physical Slot: 0-2
        Flags: fast devsel
        Memory at fde00000 (64-bit, non-prefetchable) [size=16K]
        Capabilities: [50] Power Management version 3
        Capabilities: [60] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [70] Express Endpoint, MSI 00
        Capabilities: [100] Latency Tolerance Reporting

What am I still missing?
Click to expand...
After going over my configuration several times, I ended up updating the VM I was running from Ubuntu 23.10 to 24.04 LTS and suddenly the kernel drivers started working in the VM / pod! Very frustrating that I assumed the drivers were working correctly this whole time
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back