Nvidia vGPU No Mediated devices

[Adevill20]

I need help to setup a NVidia RTX A5000 vGPU. I followed the instructions listed here https://pve.proxmox.com/wiki/NVIDIA_vGPU_on_Proxmox_VE and I get the following outputs from the Nvidia drivers, but for some reason there are no Mediated devices listed. I have a supported GPU and using the latest GRID drivers. Secure boot is disabled and the Display output of the GPU is also disabled:

 

[VictorSTS]

If using kernel 6.8, take a look at this thread [1], as newer kernels use Vendor Specific VFIO instead of Mediated Devices and currently VMs and vGPU have to be configured somewhat manually instead of through the webUI. There's some code done to adapt to this change so it can be used from the webUI soon [2].

[1] https://forum.proxmox.com/threads/vgpu-with-nvidia-on-kernel-6-8.150840/
[2] https://forum.proxmox.com/threads/vgpu-with-nvidia-on-kernel-6-8.150840/post-690894

 

[Adevill20]

If using kernel 6.8, take a look at this thread [1], as newer kernels use Vendor Specific VFIO instead of Mediated Devices and currently VMs and vGPU have to be configured somewhat manually instead of through the webUI. There's some code done to adapt to this change so it can be used from the webUI soon [2].

[1] https://forum.proxmox.com/threads/vgpu-with-nvidia-on-kernel-6-8.150840/
[2] https://forum.proxmox.com/threads/vgpu-with-nvidia-on-kernel-6-8.150840/post-690894

Thank you for the information. I followed these steps but the VM still doesnt want to start. Please see the error below:

 

[VictorSTS]

Which steps did you follow exactly? Help me help you
Reboot the PVE host, as sometimes the driver/hardware just gets stuck. Then post the output of /usr/lib/nvidia/sriov-manage -e ALL so I can see all the PCI IDs of the vGPUs.
What's the output of cat /sys/bus/pci/devices/<DOMAIN>\:<BUS>\:<SLOT>.<FUNCTION>/nvidia/creatable_vgpu_types ? Using the PCI ID of one of the vGPUs.
Do you set the type of the GPU using something like echo 918 > /sys/bus/pci/devices/0000\:26\:00.4/nvidia/current_vgpu_type?
Post the vm configuration qm config VMID .

 

[Adevill20]

I followed the exact steps of the first guide (https://pve.proxmox.com/wiki/NVIDIA_vGPU_on_Proxmox_VE). Then the steps as per the instructions of this post (https://forum.proxmox.com/threads/vgpu-with-nvidia-on-kernel-6-8.150840/).

In summary:
I enabled IOMMU (bios, grub and modules). Installed the latest Nvidia drivers, the followed the instructions you posted. The GPU is seen by PVE, the VFs are seen and I set the GPU Type to 8Q for the first FV (0000:61:00.4)



VM Config:
root@pve1:/sys/bus/pci/devices/0000:61:00.4/nvidia# cat /etc/pve/qemu-server/8402.conf
agent: 1
args: -device vfio-pci,sysfsdev=/sys/bus/pci/devices/0000:61:00.4 --uuid 7396508e-201b-4911-9dd2-885be0c3c681
bios: ovmf
boot: order=scsi0
cores: 8
cpu: host,flags=+ibpb;+virt-ssbd;+amd-ssbd;+pdpe1gb;+aes
efidisk0: nas1-m2:8402/vm-8402-disk-0.qcow2,efitype=4m,pre-enrolled-keys=1,size=528K
ide0: nas1-m2:iso/virtio-win-0.1.248.iso,media=cdrom,size=715188K
machine: pc-q35-9.0
memory: 8192
meta: creation-qemu=9.0.0,ctime=1721222033
name: Sandbox4-Client1
net0: virtio=BC:24:112:85:C3,bridge=Sandbox4
numa: 1
onboot: 1
ostype: win10
parent: AsBuilt
scsi0: nas1-m2:8402/vm-8402-disk-1.qcow2,aio=native,cache=directsync,iothread=1,size=100G,ssd=1
scsihw: virtio-scsi-single
smbios1: uuid=7396508e-201b-4911-9dd2-885be0c3c681
sockets: 1
startup: up=210
tpmstate0: nas1-m2:8402/vm-8402-disk-0.raw,size=4M,version=v2.0
vga: none
vmgenid: 14bab5f9-5eaa-4ae6-ae43-e550e1e2438e
vmstatestorage: nas1-m2

[AsBuilt]
agent: 1
args: -device vfio-pci,sysfsdev=/sys/bus/pci/devices/0000:61:00.4 --uuid 7396508e-201b-4911-9dd2-885be0c3c681
bios: ovmf
boot: order=scsi0
cores: 8
cpu: host,flags=+md-clear;+ibpb;+virt-ssbd;+amd-ssbd;+amd-no-ssb;+pdpe1gb;+aes
efidisk0: nas1-m2:8402/vm-8402-disk-0.qcow2,efitype=4m,pre-enrolled-keys=1,size=528K
ide0: nas1-m2:iso/virtio-win-0.1.248.iso,media=cdrom,size=715188K
machine: pc-q35-9.0
memory: 8192
meta: creation-qemu=9.0.0,ctime=1721222033
name: Sandbox4-Client1
net0: virtio=BC:24:112:85:C3,bridge=Sandbox4
numa: 1
onboot: 1
ostype: win10
scsi0: nas1-m2:8402/vm-8402-disk-1.qcow2,aio=native,cache=directsync,iothread=1,size=100G,ssd=1
scsihw: virtio-scsi-single
smbios1: uuid=7396508e-201b-4911-9dd2-885be0c3c681
snaptime: 1721224793
sockets: 1
startup: up=210
vga: virtio
vmgenid: 14bab5f9-5eaa-4ae6-ae43-e550e1e2438e

 

[Adevill20]

Found my error: I added 2 "--" to the UUID part of the VM config

After fixing it the VM booted and the vGPU was recognized. Thank you for the help!!