PCIe Passthrough of Atheros AR9280

[StackUnderflow]

I followed this exactly:

https://pve.proxmox.com/wiki/Pci_passthrough

I get this error when trying to spin up the VM.

kvm: -device vfio-pci,host=02:00.0,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0: vfio error: 0000:02:00.0: failed to add PCI capability 0x11[0x70]@0x90: table & pba overlap, or they don't fit in BARs, or don't align
TASK ERROR: start failed: command '/usr/bin/kvm -id 100 -name router -chardev 'socket,id=qmp,path=/var/run/qemu-server/100.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -pidfile /var/run/qemu-server/100.pid -daemonize -smbios 'type=1,uuid=1aabbb3d-7d43-4912-aa32-5af891f5c99c' -smp '2,sockets=1,cores=2,maxcpus=2' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' -vga std -vnc unix:/var/run/qemu-server/100.vnc,x509,password -cpu host,+kvm_pv_unhalt,+kvm_pv_eoi -m 1024 -readconfig /usr/share/qemu-server/pve-q35.cfg -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' -device 'vfio-pci,host=02:00.0,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:8ea536f7fb5' -drive 'if=none,id=drive-ide2,media=cdrom,aio=threads' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200' -drive 'file=/dev/vmdata/vm-100-disk-2,if=none,id=drive-virtio0,format=raw,cache=none,aio=native,detect-zeroes=on' -device 'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,bootindex=100' -netdev 'type=tap,id=net0,ifname=tap100i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=02:AB:DF:D7:9F:9E,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300' -netdev 'type=tap,id=net1,ifname=tap100i1,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=92:04:42:36:17:94,netdev=net1,bus=pci.0,addr=0x13,id=net1,bootindex=301' -netdev 'type=tap,id=net2,ifname=tap100i2,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=66:91:FF:5D:82:7E,netdev=net2,bus=pci.0,addr=0x14,id=net2,bootindex=302' -netdev 'type=tap,id=net3,ifname=tap100i3,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=8E:7F:E4:6E:CD:88,netdev=net3,bus=pci.0,addr=0x15,id=net3,bootindex=303' -machine 'type=q35'' failed: exit code 1

This is what I put into the vm conf

hostpci0: 02:00.0,pcie=1
machine: q35

This is what happens when I run find /sys/kernel/iommu_groups/ -type l

/sys/kernel/iommu_groups/55/devices/0000:ff:0f.4
/sys/kernel/iommu_groups/55/devices/0000:ff:0f.2
/sys/kernel/iommu_groups/55/devices/0000:ff:0f.0
/sys/kernel/iommu_groups/55/devices/0000:ff:0f.5
/sys/kernel/iommu_groups/55/devices/0000:ff:0f.3
/sys/kernel/iommu_groups/55/devices/0000:ff:0f.1
/sys/kernel/iommu_groups/55/devices/0000:ff:0f.6
/sys/kernel/iommu_groups/17/devices/0000:00:1e.0
/sys/kernel/iommu_groups/45/devices/0000:80:05.4
/sys/kernel/iommu_groups/45/devices/0000:80:05.2
/sys/kernel/iommu_groups/45/devices/0000:80:05.0
/sys/kernel/iommu_groups/35/devices/0000:7f:0d.1
/sys/kernel/iommu_groups/35/devices/0000:7f:0d.6
/sys/kernel/iommu_groups/35/devices/0000:7f:0d.2
/sys/kernel/iommu_groups/35/devices/0000:7f:0d.0
/sys/kernel/iommu_groups/35/devices/0000:7f:0d.3
/sys/kernel/iommu_groups/7/devices/0000:00:11.0
/sys/kernel/iommu_groups/25/devices/0000:0d:00.0
/sys/kernel/iommu_groups/53/devices/0000:ff:0d.3
/sys/kernel/iommu_groups/53/devices/0000:ff:0d.1
/sys/kernel/iommu_groups/53/devices/0000:ff:0d.6
/sys/kernel/iommu_groups/53/devices/0000:ff:0d.2
/sys/kernel/iommu_groups/53/devices/0000:ff:0d.0
/sys/kernel/iommu_groups/15/devices/0000:00:1c.7
/sys/kernel/iommu_groups/43/devices/0000:80:03.2
/sys/kernel/iommu_groups/33/devices/0000:7f:0b.0
/sys/kernel/iommu_groups/33/devices/0000:7f:0b.3
/sys/kernel/iommu_groups/5/devices/0000:00:03.2
/sys/kernel/iommu_groups/23/devices/0000:0b:00.0
/sys/kernel/iommu_groups/51/devices/0000:ff:0b.0
/sys/kernel/iommu_groups/51/devices/0000:ff:0b.3
/sys/kernel/iommu_groups/13/devices/0000:00:1c.4
/sys/kernel/iommu_groups/41/devices/0000:80:03.0
/sys/kernel/iommu_groups/31/devices/0000:7f:09.4
/sys/kernel/iommu_groups/31/devices/0000:7f:09.0
/sys/kernel/iommu_groups/31/devices/0000:7f:09.3
/sys/kernel/iommu_groups/3/devices/0000:00:02.0
/sys/kernel/iommu_groups/21/devices/0000:06:00.0
/sys/kernel/iommu_groups/21/devices/0000:06:00.1
/sys/kernel/iommu_groups/21/devices/0000:05:02.0
/sys/kernel/iommu_groups/11/devices/0000:00:1c.1
/sys/kernel/iommu_groups/1/devices/0000:00:01.0
/sys/kernel/iommu_groups/58/devices/0000:ff:13.0
/sys/kernel/iommu_groups/58/devices/0000:ff:13.5
/sys/kernel/iommu_groups/58/devices/0000:ff:13.1
/sys/kernel/iommu_groups/58/devices/0000:ff:13.6
/sys/kernel/iommu_groups/58/devices/0000:ff:13.4
/sys/kernel/iommu_groups/48/devices/0000:ff:08.3
/sys/kernel/iommu_groups/48/devices/0000:ff:08.4
/sys/kernel/iommu_groups/48/devices/0000:ff:08.0
/sys/kernel/iommu_groups/38/devices/0000:7f:10.1
/sys/kernel/iommu_groups/38/devices/0000:7f:10.6
/sys/kernel/iommu_groups/38/devices/0000:7f:10.4
/sys/kernel/iommu_groups/38/devices/0000:7f:10.2
/sys/kernel/iommu_groups/38/devices/0000:7f:10.0
/sys/kernel/iommu_groups/38/devices/0000:7f:10.7
/sys/kernel/iommu_groups/38/devices/0000:7f:10.5
/sys/kernel/iommu_groups/38/devices/0000:7f:10.3
/sys/kernel/iommu_groups/28/devices/0000:11:00.0
/sys/kernel/iommu_groups/56/devices/0000:ff:10.7
/sys/kernel/iommu_groups/56/devices/0000:ff:10.5
/sys/kernel/iommu_groups/56/devices/0000:ff:10.3
/sys/kernel/iommu_groups/56/devices/0000:ff:10.1
/sys/kernel/iommu_groups/56/devices/0000:ff:10.6
/sys/kernel/iommu_groups/56/devices/0000:ff:10.4
/sys/kernel/iommu_groups/56/devices/0000:ff:10.2
/sys/kernel/iommu_groups/56/devices/0000:ff:10.0
/sys/kernel/iommu_groups/18/devices/0000:00:1f.3
/sys/kernel/iommu_groups/18/devices/0000:00:1f.2
/sys/kernel/iommu_groups/18/devices/0000:00:1f.0
/sys/kernel/iommu_groups/46/devices/0000:81:00.0
/sys/kernel/iommu_groups/36/devices/0000:7f:0e.0
/sys/kernel/iommu_groups/36/devices/0000:7f:0e.1
/sys/kernel/iommu_groups/8/devices/0000:00:16.1
/sys/kernel/iommu_groups/8/devices/0000:00:16.0
/sys/kernel/iommu_groups/26/devices/0000:0f:00.0
/sys/kernel/iommu_groups/26/devices/0000:0e:00.0
/sys/kernel/iommu_groups/54/devices/0000:ff:0e.1
/sys/kernel/iommu_groups/54/devices/0000:ff:0e.0
/sys/kernel/iommu_groups/16/devices/0000:00:1d.0
/sys/kernel/iommu_groups/44/devices/0000:80:03.3
/sys/kernel/iommu_groups/34/devices/0000:7f:0c.7
/sys/kernel/iommu_groups/34/devices/0000:7f:0c.3
/sys/kernel/iommu_groups/34/devices/0000:7f:0c.1
/sys/kernel/iommu_groups/34/devices/0000:7f:0c.6
/sys/kernel/iommu_groups/34/devices/0000:7f:0c.2
/sys/kernel/iommu_groups/34/devices/0000:7f:0c.0
/sys/kernel/iommu_groups/6/devices/0000:00:05.4
/sys/kernel/iommu_groups/6/devices/0000:00:05.2
/sys/kernel/iommu_groups/6/devices/0000:00:05.0
/sys/kernel/iommu_groups/24/devices/0000:0c:00.0
/sys/kernel/iommu_groups/52/devices/0000:ff:0c.2
/sys/kernel/iommu_groups/52/devices/0000:ff:0c.0
/sys/kernel/iommu_groups/52/devices/0000:ff:0c.7
/sys/kernel/iommu_groups/52/devices/0000:ff:0c.3
/sys/kernel/iommu_groups/52/devices/0000:ff:0c.1
/sys/kernel/iommu_groups/52/devices/0000:ff:0c.6
/sys/kernel/iommu_groups/14/devices/0000:00:1c.6
/sys/kernel/iommu_groups/42/devices/0000:80:03.1
/sys/kernel/iommu_groups/32/devices/0000:7f:0a.2
/sys/kernel/iommu_groups/32/devices/0000:7f:0a.0
/sys/kernel/iommu_groups/32/devices/0000:7f:0a.3
/sys/kernel/iommu_groups/32/devices/0000:7f:0a.1
/sys/kernel/iommu_groups/4/devices/0000:00:03.0
/sys/kernel/iommu_groups/22/devices/0000:05:04.0
/sys/kernel/iommu_groups/22/devices/0000:08:00.1
/sys/kernel/iommu_groups/22/devices/0000:08:00.0
/sys/kernel/iommu_groups/50/devices/0000:ff:0a.3
/sys/kernel/iommu_groups/50/devices/0000:ff:0a.1
/sys/kernel/iommu_groups/50/devices/0000:ff:0a.2
/sys/kernel/iommu_groups/50/devices/0000:ff:0a.0
/sys/kernel/iommu_groups/12/devices/0000:00:1c.3
/sys/kernel/iommu_groups/40/devices/0000:7f:13.1
/sys/kernel/iommu_groups/40/devices/0000:7f:13.6
/sys/kernel/iommu_groups/40/devices/0000:7f:13.4
/sys/kernel/iommu_groups/40/devices/0000:7f:13.0
/sys/kernel/iommu_groups/40/devices/0000:7f:13.5
/sys/kernel/iommu_groups/30/devices/0000:7f:08.4
/sys/kernel/iommu_groups/30/devices/0000:7f:08.0
/sys/kernel/iommu_groups/30/devices/0000:7f:08.3
/sys/kernel/iommu_groups/2/devices/0000:00:01.1
/sys/kernel/iommu_groups/20/devices/0000:04:00.0
/sys/kernel/iommu_groups/49/devices/0000:ff:09.4
/sys/kernel/iommu_groups/49/devices/0000:ff:09.0
/sys/kernel/iommu_groups/49/devices/0000:ff:09.3
/sys/kernel/iommu_groups/10/devices/0000:00:1c.0
/sys/kernel/iommu_groups/39/devices/0000:7f:11.0
/sys/kernel/iommu_groups/29/devices/0000:12:00.0
/sys/kernel/iommu_groups/0/devices/0000:00:00.0
/sys/kernel/iommu_groups/57/devices/0000:ff:11.0
/sys/kernel/iommu_groups/19/devices/0000:02:00.0
/sys/kernel/iommu_groups/47/devices/0000:82:00.0
/sys/kernel/iommu_groups/37/devices/0000:7f:0f.5
/sys/kernel/iommu_groups/37/devices/0000:7f:0f.3
/sys/kernel/iommu_groups/37/devices/0000:7f:0f.1
/sys/kernel/iommu_groups/37/devices/0000:7f:0f.6
/sys/kernel/iommu_groups/37/devices/0000:7f:0f.4
/sys/kernel/iommu_groups/37/devices/0000:7f:0f.2
/sys/kernel/iommu_groups/37/devices/0000:7f:0f.0
/sys/kernel/iommu_groups/9/devices/0000:00:1a.0
/sys/kernel/iommu_groups/27/devices/0000:10:00.0

How do I fix this so the VM boots again?

 

[wolfgang]

Hi,

do you use ovmf as bios?

 

[Idar Lund]

I have the same error on the same card.
I have tried both ovmf and seabios. The same error message appears.
As a test I used an Intel card in the same pci slot and that worked. However, this is not a suitable solution since the Intel card can only act as a client not AP.

lspci:

06:00.0 Network controller [0280]: Qualcomm Atheros AR928X Wireless Network Adapter (PCI-Express) [168c:002a] (rev 01) 
Subsystem: Qualcomm Atheros AR928X Wireless Network Adapter (PCI-Express) [168c:3099] 
Kernel driver in use: vfio-pci 
Kernel modules: ath9k

Error mesage:

kvm: -device vfio-pci,host=0000:06:00.0,id=hostpci0,bus=pci.0,addr=0x10: vfio 0000:06:00.0: failed to add PCI capability 0x11[0x70]@0x90: table & pba overlap, or they don't fit in BARs, or don't align

/etc/modprobe.d/wireless.conf:

blacklist ath9k 
options vfio-pci ids=168c:002a,168c:3099

Any ideas on how to fix this?

 

[DragonExploding]

This link may help you: https://bugzilla.kernel.org/show_bug.cgi?id=202055#c47

 

[Idar Lund]

Thanks! That worked. For the record and for people landing on this page from google:

I only had 1 pci device passtrough so it was named hostpci0 in the "/etc/pve/local/qemu-server/100.conf" config file, like this:
hostpci0: 06:00.0
Added the following to the bottom solved the pci passtrough:
args: -set device.hostpci0.x-msix-relocation=bar2

 

[leeivan]

If have several pci devices passthrough , how to write the code "args: -set "?

 

[leesteken]

If have several pci devices passthrough , how to write the code "args: -set "?

Add the line to the configuration file of the VM.

 

[jimklimov]

Super! Thanks for this post. I actually finally signed up just to say thanks - similar arg helped me pass-through an NVMe as a PCIe device. It was salvaged from a deceased laptop, and allowed to use its previously installed system "as is" on my VM rig.

 

[babsz02]

Thanks !
It worked for me.
I was trying to connect a M2 NVME ssd with a Jeyi NVME PCI x4 card in PCI Passtrough on an OpenMediaVault VM !
Thanks