I am using a small system utilizing an AMD EPYC Embedded 3451 on an asrockrack mobo. System has been rock solid for well over a year, with a truenas vm running (among other vms / containers). The truenas vm had the board's integrated LSI controller passed to it in order to directly access the attached drives. Today, I added an intel arc a-series card to the system and decided to upgrade to proxmox 9. The system is a test bed, so I installed proxmox from virtual media fresh, blowing away the old install entirely. After the install completed, I attempted to build a couple of vms to test that pass through was still working correctly and to see if I could pass through the new GPU to a VM. Doing this, I have found that I am no longer to pass through any pci devices. I've tried the gpu and disk controller, but no luck.
The error I am receiving when I try to boot a vm with a pass through device allocated is :
I have verified that all of the devices i'm trying to pass are in their own iommu groups.
modules-load/vfio.conf:
lsmod output :
config of vm used to test:
I've had no luck searching for a post regarding this sort of error on an AMD system, the only other post i found that referenced the same type of error i'm seeing was on an intel system and was apparently tied to a kernel problem that was patched / mitigated:
Can anyone offer me some insight into where I my look next to solve my problem? I would gladly accept that i'm messing up something on the gpu pass through, but passing the disk controller / disks was working fine prior to the move to v9, and the error i'm seeing in every case is the same form.
The error I am receiving when I try to boot a vm with a pass through device allocated is :
[ 91.130891] vfio-pci 0000:4b:00.0: Firmware has requested this device have a 1:1 IOMMU mapping, rejecting configuring the device without a 1:1 mapping. Contact your platform vendor.I have verified that all of the devices i'm trying to pass are in their own iommu groups.
Code:
pvesh get /nodes/pve/hardware/pci --pci-class-blacklist "" --noborder
class device id iommugroup vendor device_name mde
0x010700 0x0097 0000:4b:00.0 57 0x1000 SAS3008 PCI-Express Fusion-MPT SAS-3
0x010802 0x540a 0000:01:00.0 20 0xc0a9 P2 [Nick P2] / P3 / P3 Plus NVMe PCIe SSD (DRAM-less)
0x010802 0x2525 0000:47:00.0 55 0x8086 Optane NVME SSD P1600X Series
0x010802 0x2525 0000:49:00.0 56 0x8086 Optane NVME SSD P1600X Series
0x020000 0x15ff 0000:06:00.0 21 0x8086 Ethernet Controller X710 for 10GBASE-T
0x020000 0x15ff 0000:06:00.1 22 0x8086 Ethernet Controller X710 for 10GBASE-T
0x020000 0x1533 0000:08:00.0 23 0x8086 I210 Gigabit Network Connection
0x020000 0x1533 0000:0b:00.0 25 0x8086 I210 Gigabit Network Connection
0x020000 0x1458 0000:0d:00.4 31 0x1022 XGMAC 10GbE Controller
0x020000 0x1458 0000:0d:00.5 32 0x1022 XGMAC 10GbE Controller
0x030000 0x2000 0000:0a:00.0 24 0x1a03 ASPEED Graphics Family
0x030000 0x56a6 0000:43:00.0 48 0x8086 DG2 [Arc A310]
0x040300 0x4f92 0000:44:00.0 49 0x8086 DG2 Audio Controller
0x060000 0x1450 0000:00:00.0 33 0x1022 Family 17h (Models 00h-0fh) Root Complex
0x060000 0x1452 0000:00:01.0 0 0x1022 Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
0x060000 0x1452 0000:00:02.0 4 0x1022 Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
0x060000 0x1452 0000:00:03.0 5 0x1022 Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
0x060000 0x1452 0000:00:04.0 12 0x1022 Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
0x060000 0x1452 0000:00:07.0 13 0x1022 Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
0x060000 0x1452 0000:00:08.0 15 0x1022 Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
0x060000 0x1460 0000:00:18.0 18 0x1022 Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0
0x060000 0x1461 0000:00:18.1 18 0x1022 Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1
0x060000 0x1462 0000:00:18.2 18 0x1022 Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2
0x060000 0x1463 0000:00:18.3 18 0x1022 Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3
0x060000 0x1464 0000:00:18.4 18 0x1022 Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4
0x060000 0x1465 0000:00:18.5 18 0x1022 Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5
0x060000 0x1466 0000:00:18.6 18 0x1022 Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6
0x060000 0x1467 0000:00:18.7 18 0x1022 Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7
0x060000 0x1460 0000:00:19.0 19 0x1022 Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0
0x060000 0x1461 0000:00:19.1 19 0x1022 Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1
0x060000 0x1462 0000:00:19.2 19 0x1022 Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2
0x060000 0x1463 0000:00:19.3 19 0x1022 Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3
0x060000 0x1464 0000:00:19.4 19 0x1022 Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4
0x060000 0x1465 0000:00:19.5 19 0x1022 Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5
0x060000 0x1466 0000:00:19.6 19 0x1022 Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6
0x060000 0x1467 0000:00:19.7 19 0x1022 Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7
0x060000 0x1450 0000:40:00.0 -1 0x1022 Family 17h (Models 00h-0fh) Root Complex
0x060000 0x1452 0000:40:01.0 34 0x1022 Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
0x060000 0x1452 0000:40:02.0 36 0x1022 Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
0x060000 0x1452 0000:40:03.0 37 0x1022 Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
0x060000 0x1452 0000:40:04.0 40 0x1022 Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
0x060000 0x1452 0000:40:07.0 41 0x1022 Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
0x060000 0x1452 0000:40:08.0 43 0x1022 Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
0x060100 0x790e 0000:00:14.3 17 0x1022 FCH LPC Bridge
0x060400 0x1453 0000:00:01.1 1 0x1022 Family 17h (Models 00h-0fh) PCIe GPP Bridge
0x060400 0x1453 0000:00:01.2 2 0x1022 Family 17h (Models 00h-0fh) PCIe GPP Bridge
0x060400 0x1453 0000:00:01.3 3 0x1022 Family 17h (Models 00h-0fh) PCIe GPP Bridge
0x060400 0x1453 0000:00:03.1 6 0x1022 Family 17h (Models 00h-0fh) PCIe GPP Bridge
0x060400 0x1453 0000:00:03.2 7 0x1022 Family 17h (Models 00h-0fh) PCIe GPP Bridge
0x060400 0x1453 0000:00:03.3 8 0x1022 Family 17h (Models 00h-0fh) PCIe GPP Bridge
0x060400 0x1453 0000:00:03.4 9 0x1022 Family 17h (Models 00h-0fh) PCIe GPP Bridge
0x060400 0x1453 0000:00:03.5 10 0x1022 Family 17h (Models 00h-0fh) PCIe GPP Bridge
0x060400 0x1453 0000:00:03.6 11 0x1022 Family 17h (Models 00h-0fh) PCIe GPP Bridge
0x060400 0x1454 0000:00:07.1 14 0x1022 Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B
0x060400 0x1454 0000:00:08.1 16 0x1022 Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B
0x060400 0x1150 0000:09:00.0 24 0x1a03 AST1150 PCI-to-PCI Bridge
0x060400 0x1453 0000:40:01.1 35 0x1022 Family 17h (Models 00h-0fh) PCIe GPP Bridge
0x060400 0x1453 0000:40:03.1 38 0x1022 Family 17h (Models 00h-0fh) PCIe GPP Bridge
0x060400 0x1453 0000:40:03.3 39 0x1022 Family 17h (Models 00h-0fh) PCIe GPP Bridge
0x060400 0x1454 0000:40:07.1 42 0x1022 Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B
0x060400 0x1454 0000:40:08.1 44 0x1022 Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B
0x060400 0x4fa1 0000:41:00.0 45 0x8086
0x060400 0x4fa4 0000:42:01.0 46 0x8086
0x060400 0x4fa4 0000:42:04.0 47 0x8086
0x060400 0x2824 0000:45:00.0 50 0x1b21 ASM2824 PCIe Gen3 Packet Switch
0x060400 0x2824 0000:46:00.0 51 0x1b21 ASM2824 PCIe Gen3 Packet Switch
0x060400 0x2824 0000:46:04.0 52 0x1b21 ASM2824 PCIe Gen3 Packet Switch
0x060400 0x2824 0000:46:08.0 53 0x1b21 ASM2824 PCIe Gen3 Packet Switch
0x060400 0x2824 0000:46:0c.0 54 0x1b21 ASM2824 PCIe Gen3 Packet Switch
0x080600 0x1451 0000:00:00.2 -1 0x1022 Family 17h (Models 00h-0fh) I/O Memory Management Unit
0x080600 0x1451 0000:40:00.2 -1 0x1022 Family 17h (Models 00h-0fh) I/O Memory Management Unit
0x0c0330 0x145f 0000:0c:00.3 28 0x1022 Zeppelin USB 3.0 xHCI Compliant Host Controller
0x0c0330 0x145f 0000:4c:00.3 60 0x1022 Zeppelin USB 3.0 xHCI Compliant Host Controller
0x0c0500 0x790b 0000:00:14.0 17 0x1022 FCH SMBus Controller
0x108000 0x1456 0000:0c:00.2 27 0x1022 Family 17h (Models 00h-0fh) Platform Security Processor (PSP) 3.0 Device
0x108000 0x1468 0000:0d:00.1 30 0x1022 Zeppelin Cryptographic Coprocessor NTBCCP
0x108000 0x1456 0000:4c:00.2 59 0x1022 Family 17h (Models 00h-0fh) Platform Security Processor (PSP) 3.0 Device
0x108000 0x1468 0000:4d:00.1 62 0x1022 Zeppelin Cryptographic Coprocessor NTBCCP
0x130000 0x145a 0000:0c:00.0 26 0x1022 Zeppelin/Raven/Raven2 PCIe Dummy Function
0x130000 0x1455 0000:0d:00.0 29 0x1022 Zeppelin/Renoir PCIe Dummy Function
0x130000 0x145a 0000:4c:00.0 58 0x1022 Zeppelin/Raven/Raven2 PCIe Dummy Function
0x130000 0x1455 0000:4d:00.0 61 0x1022 Zeppelin/Renoir PCIe Dummy Functionmodules-load/vfio.conf:
Code:
root@pve:~# cat /etc/modules-load.d/vfio.conf
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfdlsmod output :
Code:
root@pve:~# lsmod | grep vfio
vfio_pci 16384 0
vfio_pci_core 86016 1 vfio_pci
irqbypass 12288 2 vfio_pci_core,kvm
vfio_iommu_type1 49152 0
vfio 65536 4 vfio_pci_core,vfio_iommu_type1,vfio_pci
iommufd 110592 1 vfioconfig of vm used to test:
Code:
qm config 902
agent: 1
bios: ovmf
boot: order=scsi0;ide2;net0
cores: 4
cpu: x86-64-v2-AES
efidisk0: local-lvm:vm-902-disk-0,efitype=4m,pre-enrolled-keys=1,size=4M
hostpci0: 0000:4b:00
ide2: local:iso/ubuntu-24.04.3-live-server-amd64.iso,media=cdrom,size=3226020K
machine: q35
memory: 16384
meta: creation-qemu=10.0.2,ctime=1762243673
name: gpu-test-pass
net0: virtio=BC:24:11:F3:31:19,bridge=vmbr0
numa: 0
ostype: l26
scsi0: local-lvm:vm-902-disk-1,iothread=1,size=64G,ssd=1
scsihw: virtio-scsi-single
smbios1: uuid=656c1a31-ff99-426e-a897-fc37ae06b42d
sockets: 1
vmgenid: 5752062d-9274-4710-8742-39dd151b6178Can anyone offer me some insight into where I my look next to solve my problem? I would gladly accept that i'm messing up something on the gpu pass through, but passing the disk controller / disks was working fine prior to the move to v9, and the error i'm seeing in every case is the same form.
Last edited: