RTX 5080 VFIO passthrough fails on Proxmox VE 9.1 with error getting device from group 13: No such device

[temir_ast]

Hello,


I am trying to pass through an NVIDIA RTX 5080 (GB203) to a VM on Proxmox VE 9.1, but the VM fails to start. I would appreciate any advice from anyone who has seen something similar.

Environment​

• Host: Proxmox VE 9.1
• [B]Tested kernels:[/B] 6.17.13-2-pve and 6.14.11-6-pve
• [B]CPU:[/B] Intel with VT-d enabled
• [B]Kernel parameters:[/B] intel_iommu=on iommu=pt
• [B]GPU:[/B] Palit GeForce RTX 5080
  • VGA: 10de:2c02
  • Audio: 10de:22e9

IOMMU group​

The GPU is in IOMMU group 13, and the group contains only these two devices:

• 0000:02:00.0 VGA compatible controller
• 0000:02:00.1 Audio device

Both functions are bound to vfio-pci.


lspci -nnk -s 02:00 shows:



[CODE]02:00.0 VGA compatible controller [0300]: NVIDIA Corporation GB203 [GeForce RTX 5080] [10de:2c02] (rev a1)
Subsystem: Palit Microsystems Inc. Device [1569:f322]
Kernel driver in use: vfio-pci
Kernel modules: nvidiafb, nouveau[/CODE]

02:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:22e9] (rev a1)
        Subsystem: NVIDIA Corporation Device [10de:0000]
        Kernel driver in use: vfio-pci
        Kernel modules: snd_hda_intel

VM configuration​

I tested the VM with OVMF and q35. Current config:

agent: 1
bios: ovmf
machine: q35
cpu: host
cores: 4
sockets: 1
memory: 8192
hostpci0: 0000:02:00,pcie=1
vga: std
boot: order=virtio0;net0

I also tested:

• hostpci0: 0000:02:00,pcie=1,x-vga=1 with vga: none
• separate function passthrough
• adding an EFI disk
• removing PCI mapping and using direct PCI address only

Error​

Every time I start the VM, I get:

error writing '1' to '/sys/bus/pci/devices/0000:02:00.0/reset': Inappropriate ioctl for device
failed to reset PCI device '0000:02:00.0', but trying to continue as not all devices need a reset
kvm: -device vfio-pci,host=0000:02:00.0,id=hostpci0.0,bus=ich9-pcie-port-1,addr=0x0.0,multifunction=on: vfio 0000:02:00.0: error getting device from group 13: No such device
Verify all devices in group 13 are bound to vfio-<bus> or pci-stub and not already in use
start failed: QEMU exited with code 1

What I already checked​

• IOMMU group is clean and contains only the GPU and its audio function
• both functions are bound to vfio-pci
• tested on both 6.17.13-2-pve and 6.14.11-6-pve
• tested with and without x-vga=1
• tested with vga: none and vga: std
• added efidisk0
• blacklisted host GPU-related modules where applicable

Additional information​

On the host:




CONFIG_VFIO_DEVICE_CDEV=y



Loaded modules:




vfio_pci 16384 0
vfio_pci_core 86016 1 vfio_pci
irqbypass 12288 2 vfio_pci_core,kvm
vfio_iommu_type1 49152 0
vfio 65536 4 vfio_pci_core,vfio_iommu_type1,vfio_pci
iommufd 110592 1 vfio



/dev/vfio/13 exists, but lsof /dev/vfio/13 shows nothing.


This makes me wonder whether this is related to:

• RTX 5080 / Blackwell support
• iommufd vs legacy VFIO group handling
• motherboard BIOS / firmware behavior
• DMA protection / kernel DMA protection settings in BIOS

Questions​

Has anyone successfully passed through an RTX 5080 on Proxmox VE 9.x?


Does this look like a known iommufd / VFIO issue rather than a normal passthrough misconfiguration?


Would you recommend checking BIOS options such as:

• VT-d / IOMMU = Enabled
• Kernel DMA Protection = Disabled
• Pre-boot DMA Protection
• BIOS update to latest version

Any help or suggestions would be greatly appreciated.


Thanks.

 

[Soepgroenten]

I got 100% exactly the same issue also with an RTX5080

i got mine in IOMMU group 15

lsof /dev/vfio/15 ends up blank for me too.

 

[dcsapak]

there is already an open bug on https://bugzilla.proxmox.com/show_bug.cgi?id=7374 but we don't currently know what's the underlying reason. (and we don't have the matching hardware here to try to bisect the kernel)

are you by any chance using a gigabyte mainboard and/or gpu?

 

[Soepgroenten]

there is already an open bug on https://bugzilla.proxmox.com/show_bug.cgi?id=7374 but we don't currently know what's the underlying reason. (and we don't have the matching hardware here to try to bisect the kernel)

are you by any chance using a gigabyte mainboard and/or gpu?

Nope MSI main board and VGA in combination with a core 9 ultra 285h (if that makes sense).

From what I've read, the VGA isn't able to wake up from d3cold state because the VGA is claimed before it can be used by the VM or container.