[SOLVED] mdev types shows data listing nvidia vgpus, but proxmox 8.2 'add pcie device' selector is empty?

[dooferorg]

I have (after some caveats and help from https://gitlab.com/polloloco/vgpu-proxmox and https://gitlab.com/polloloco/vgpu-proxmox/-/merge_requests/9 ) got my Tesla P4 recognized within a Proxmox 8.2 / kernel 6.8 server.

nvidia-smi returns information about the card as expected and mdevctl types lists a variety of 'nvidia-??' devices.

However, when it comes to the selector within Proxmox for managing the hardware of a Windows VM, the selector is blank and I'm unable to select any of the nvidia devices.

Is this still a bug? I saw references to it being an issue in 8.1.4.

 

[Zackptg5]

nvidia drivers haven't been updated for 6.8 kernel yet. You'll either need to wait for them to update it (who knows when that'll be) or pin 6.5 kernel and boot with it

 

[dooferorg]

nvidia drivers haven't been updated for 6.8 kernel yet. You'll either need to wait for them to update it (who knows when that'll be) or pin 6.5 kernel and boot with it

I don't think you really read what I wrote .. - I have them compiled just fine with a patch obtained from here: https://gitlab.com/polloloco/vgpu-proxmox/-/merge_requests/9 with the author's help this morning.

nvidia-smi returns

Tue May  7 12:22:44 2024
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.161.05             Driver Version: 535.161.05   CUDA Version: N/A      |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  Tesla P4                       On  | 00000000:04:00.0 Off |                    0 |
| N/A   30C    P8              10W /  75W |     31MiB /  7680MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|  No running processes found                                                           |
+---------------------------------------------------------------------------------------+

nvidia-smi vpu returns:

Tue May  7 12:22:47 2024
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 535.161.05             Driver Version: 535.161.05                |
|---------------------------------+------------------------------+------------+
| GPU  Name                       | Bus-Id                       | GPU-Util   |
|      vGPU ID     Name           | VM ID     VM Name            | vGPU-Util  |
|=================================+==============================+============|
|   0  Tesla P4                   | 00000000:04:00.0             |   0%       |
+---------------------------------+------------------------------+------------+

mdevctl types:

0000:04:00.0
  nvidia-156
    Available instances: 12
    Device API: vfio-pci
    Name: GRID P40-2B
    Description: num_heads=4, frl_config=45, framebuffer=2048M, max_resolution=5120x2880, max_instance=12
  nvidia-215
    Available instances: 12
    Device API: vfio-pci
    Name: GRID P40-2B4
    Description: num_heads=4, frl_config=45, framebuffer=2048M, max_resolution=5120x2880, max_instance=12
  nvidia-241
    Available instances: 24
    Device API: vfio-pci
    Name: GRID P40-1B4
    Description: num_heads=4, frl_config=45, framebuffer=1024M, max_resolution=5120x2880, max_instance=24
  nvidia-46
    Available instances: 24
    Device API: vfio-pci
    Name: GRID P40-1Q
    Description: num_heads=4, frl_config=60, framebuffer=1024M, max_resolution=5120x2880, max_instance=24
  nvidia-47
    Available instances: 12
    Device API: vfio-pci
    Name: GRID P40-2Q
    Description: num_heads=4, frl_config=60, framebuffer=2048M, max_resolution=7680x4320, max_instance=12
  nvidia-48
    Available instances: 8
    Device API: vfio-pci
    Name: GRID P40-3Q

....

It's the GUI that doesn't display it.

 

[dooferorg]

Edit.. I realized I was doing it wrong after I manually set the device via the command line as per https://pve.proxmox.com/wiki/NVIDIA_vGPU_on_Proxmox_VE

I should have selected 'Raw device', selected the Tesla card and *then* the MDev Type selector is populated/displayed.

I was then able to boot up and install guest drivers.

Screenshot attached in case it helps anyone else in future.

 

[dcsapak]

if you configured a 'mapped device' did you check the 'mediated devices' checkbox on the configuration when you selected the devices?

 

[dooferorg]

if you configured a 'mapped device' did you check the 'mediated devices' checkbox on the configuration when you selected the devices?

No, I was admittedly confused what I should be seeing under 'mapped devices' I thought those were what should display the list of nvidia devices. I've come to learn that 'mdev' is 'mediated device' not 'mapped device' .. too many Ms

 

[dcsapak]

No, I was admittedly confused what I should be seeing under 'mapped devices' I thought those were what should display the list of nvidia devices. I've come to learn that 'mdev' is 'mediated device' not 'mapped device' .. too many Ms

ok so it was probably that checkbox missing, then you'd have seen the types. but point taken, the 'MDev' text is probably not that helpful now that we have 'mapped' and 'mediated' devices

 

[dsenela]

ok so it was probably that checkbox missing, then you'd have seen the types. but point taken, the 'MDev' text is probably not that helpful now that we have 'mapped' and 'mediated' devices

I was having same issue. This information has helped me after countless hours trying to figure this out. Conflicting information is scattered all over different forums on net.