Hi,
I'm trying to passthrough an NVIDIA GPU to a Windows 11 VM on an older AMD platform and I’m hitting a VFIO / IOMMU issue on Proxmox VE 8 with all recent kernels I tried.
Hardware:
- Mainboard / chipset: HP ProLiant DL380 Gen9
- CPU: 32 x AMD Opteron(tm) Processor 6376
- GPU for passthrough: NVIDIA GeForce GTX 1050 Ti (10de:1c82, audio 10de:0fb9)
- Host OS: Proxmox VE 8.x (Debian 12)
IOMMU is enabled in BIOS and clearly detected by the kernel:
`journalctl -b 0 | grep -i iommu`
shows (latest boot):
Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.2.16-6-pve root=/dev/mapper/pve-root ro quiet amd_iommu=on
AGP: Please enable the IOMMU option in the BIOS setup
iommu: Default domain type: Translated
iommu: DMA domain TLB invalidation policy: lazy mode
pci 0000:05:00.0: Adding to iommu group 15
pci 0000:05:00.1: Adding to iommu group 15
...
pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
pci 0000:40:00.2: AMD-Vi: Found IOMMU cap 0x40
IOMMU groups are present and the GPU + audio are alone in group 15.
Both functions are bound to vfio-pci:
`lspci -nnk | grep -A3 "05:00"`
05:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP107 [GeForce GTX 1050 Ti] [10de:1c82] (rev a1)
Kernel driver in use: vfio-pci
Kernel modules: nvidiafb, nouveau
05:00.1 Audio device [0403]: NVIDIA Corporation GP107GL High Definition Audio Controller [10de:0fb9] (rev a1)
Kernel driver in use: vfio-pci
Kernel modules: snd_hda_intel
`/etc/modprobe.d/vfio.conf`:
blacklist nouveau
blacklist nvidiafb
blacklist nvidia
blacklist radeon
options vfio-pci ids=10de:1c82,10de:0fb9
(then `update-initramfs -u -k all` and reboot).
VM config (Windows 11):
`/etc/pve/qemu-server/105.conf`:
boot: order=ide1
bios: ovmf
machine: pc-q35-10.0
cpu: host
cores: 4
memory: 2048
balloon: 0
vga: none
hostpci0: 0000:05:00,pcie=1,rombar=0
net0: e1000=BC:24:11:3B:3C:55,bridge=vmbr0
(also tried without `rombar=0`, same result)
When I start the VM, I get:
error writing '1' to '/sys/bus/pci/devices/0000:05:00.0/reset': Inappropriate ioctl for device
failed to reset PCI device '0000:05:00.0', but trying to continue as not all devices need a reset
kvm: -device vfio-pci,host=0000:05:00.0,id=hostpci0.0,bus=ich9-pcie-port-1,addr=0x0.0,multifunction=on: \
vfio 0000:05:00.0: failed to setup container for group 15: \
memory listener initialization failed: Region pc.ram: \
vfio_container_dma_map(0x..., 0x0, 0x80000000, 0x7f...,) = -22 (Invalid argument)
TASK ERROR: start failed: QEMU exited with code 1
`dmesg | grep -i vfio` around that time:
VFIO - User Level meta-driver version: 0.3
vfio-pci 0000:05:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem
wns=none
vfio_pci: add [10de:1c82[ffffffff:ffffffff]] class 0x000000/00000000
vfio_pci: add [10de:0fb9[ffffffff:ffffffff]] class 0x000000/00000000
So the card is attached to vfio-pci fine, the IOMMU groups look correct, but the DMA map for the VM RAM fails with -EINVAL.
What I already tried:
- Different PVE kernels: 6.14.11-4-pve, 6.8.12-16-pve, 6.2.16-6-pve (current).
- On newer kernels (6.8 / 6.14) I was getting:
vfio-pci 0000:05:00.0: Firmware has requested this device have a 1:1 IOMMU mapping, rejecting configuring the device without a 1:1 mapping.
so I downgraded to 6.2.16-6 where this message disappears, but instead I hit the
`vfio_container_dma_map(...)= -22` error above.
- Removed `iommu=pt` from kernel cmdline (now default domain is `Translated`).
- Reduced VM memory down to 2 GB and disabled ballooning.
- Simplified VM config (removed custom args, hidden=1 etc.).
- Verified that only 0000:05:00.0 and 0000:05:00.1 are in IOMMU group 15.
The issue feels very similar to this older thread:
https://forum.proxmox.com/threads/k...empting-to-start-a-vm-with-passthrough.67579/
where GPU passthrough worked on 5.3 but starting with 5.4 vfio_dma_map() started returning -22 on some AMD platforms.
Thanks!
I'm trying to passthrough an NVIDIA GPU to a Windows 11 VM on an older AMD platform and I’m hitting a VFIO / IOMMU issue on Proxmox VE 8 with all recent kernels I tried.
Hardware:
- Mainboard / chipset: HP ProLiant DL380 Gen9
- CPU: 32 x AMD Opteron(tm) Processor 6376
- GPU for passthrough: NVIDIA GeForce GTX 1050 Ti (10de:1c82, audio 10de:0fb9)
- Host OS: Proxmox VE 8.x (Debian 12)
IOMMU is enabled in BIOS and clearly detected by the kernel:
`journalctl -b 0 | grep -i iommu`
shows (latest boot):
Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.2.16-6-pve root=/dev/mapper/pve-root ro quiet amd_iommu=on
AGP: Please enable the IOMMU option in the BIOS setup
iommu: Default domain type: Translated
iommu: DMA domain TLB invalidation policy: lazy mode
pci 0000:05:00.0: Adding to iommu group 15
pci 0000:05:00.1: Adding to iommu group 15
...
pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
pci 0000:40:00.2: AMD-Vi: Found IOMMU cap 0x40
IOMMU groups are present and the GPU + audio are alone in group 15.
Both functions are bound to vfio-pci:
`lspci -nnk | grep -A3 "05:00"`
05:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP107 [GeForce GTX 1050 Ti] [10de:1c82] (rev a1)
Kernel driver in use: vfio-pci
Kernel modules: nvidiafb, nouveau
05:00.1 Audio device [0403]: NVIDIA Corporation GP107GL High Definition Audio Controller [10de:0fb9] (rev a1)
Kernel driver in use: vfio-pci
Kernel modules: snd_hda_intel
`/etc/modprobe.d/vfio.conf`:
blacklist nouveau
blacklist nvidiafb
blacklist nvidia
blacklist radeon
options vfio-pci ids=10de:1c82,10de:0fb9
(then `update-initramfs -u -k all` and reboot).
VM config (Windows 11):
`/etc/pve/qemu-server/105.conf`:
boot: order=ide1
bios: ovmf
machine: pc-q35-10.0
cpu: host
cores: 4
memory: 2048
balloon: 0
vga: none
hostpci0: 0000:05:00,pcie=1,rombar=0
net0: e1000=BC:24:11:3B:3C:55,bridge=vmbr0
(also tried without `rombar=0`, same result)
When I start the VM, I get:
error writing '1' to '/sys/bus/pci/devices/0000:05:00.0/reset': Inappropriate ioctl for device
failed to reset PCI device '0000:05:00.0', but trying to continue as not all devices need a reset
kvm: -device vfio-pci,host=0000:05:00.0,id=hostpci0.0,bus=ich9-pcie-port-1,addr=0x0.0,multifunction=on: \
vfio 0000:05:00.0: failed to setup container for group 15: \
memory listener initialization failed: Region pc.ram: \
vfio_container_dma_map(0x..., 0x0, 0x80000000, 0x7f...,) = -22 (Invalid argument)
TASK ERROR: start failed: QEMU exited with code 1
`dmesg | grep -i vfio` around that time:
VFIO - User Level meta-driver version: 0.3
vfio-pci 0000:05:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem
wns=nonevfio_pci: add [10de:1c82[ffffffff:ffffffff]] class 0x000000/00000000
vfio_pci: add [10de:0fb9[ffffffff:ffffffff]] class 0x000000/00000000
So the card is attached to vfio-pci fine, the IOMMU groups look correct, but the DMA map for the VM RAM fails with -EINVAL.
What I already tried:
- Different PVE kernels: 6.14.11-4-pve, 6.8.12-16-pve, 6.2.16-6-pve (current).
- On newer kernels (6.8 / 6.14) I was getting:
vfio-pci 0000:05:00.0: Firmware has requested this device have a 1:1 IOMMU mapping, rejecting configuring the device without a 1:1 mapping.
so I downgraded to 6.2.16-6 where this message disappears, but instead I hit the
`vfio_container_dma_map(...)= -22` error above.
- Removed `iommu=pt` from kernel cmdline (now default domain is `Translated`).
- Reduced VM memory down to 2 GB and disabled ballooning.
- Simplified VM config (removed custom args, hidden=1 etc.).
- Verified that only 0000:05:00.0 and 0000:05:00.1 are in IOMMU group 15.
The issue feels very similar to this older thread:
https://forum.proxmox.com/threads/k...empting-to-start-a-vm-with-passthrough.67579/
where GPU passthrough worked on 5.3 but starting with 5.4 vfio_dma_map() started returning -22 on some AMD platforms.
Thanks!