Tesla V100S and Only 8GB types show in mdevctl

[mohsenshahbazi]

Hi
we have a nvidia Tesla V100S card . we tried to install nvidia driver in esxi and got multiple profiles like Q2 4096 or another ones .
but when we switch to proxmox and using mdevctl to load types it's show Just Available Instances 3 for only 8GB memory profiles ( 8Q , 8A,8B )
and even if we assign 4GB profile like 4Q to a vm , it's got replied from mdevctl that no instance available . but we can assign this profile successfully to vm in esxi .

is there any config or something some where that limit ( mdevctl or Proxmox) only or list or promote instances for specific memory ?

 

[dcsapak]

is there any config or something some where that limit ( mdevctl or Proxmox) only or list or promote instances for specific memory ?

no, we just report what the driver tells us is available..

can you post the output of

ls -lhR  /sys/bus/pci/devices/<ID>/mdev_supported_types/

where <ID> is your pciid of the card (e.g. 0000:00:01.0)

 

[mohsenshahbazi]

Thanks for replay
here is output
ls /sys/bus/pci/devices/0000\:e1\:00.0/mdev_supported_types/ nvidia-355 nvidia-358 nvidia-361 nvidia-364 nvidia-367 nvidia-370 nvidia-356 nvidia-359 nvidia-362 nvidia-365 nvidia-368 nvidia-371 nvidia-357 nvidia-360 nvidia-363 nvidia-366 nvidia-369 nvidia-373

but output of "mdevctl types" looks like this :

nvidia-357
Available instances: 0
Device API: vfio-pci
Name: GRID V100S-4Q
Description: num_heads=4, frl_config=60, framebuffer=4096M, max_resolution=7680x4320, max_instance=8
nvidia-358
Available instances: 3
Device API: vfio-pci
Name: GRID V100S-8Q
Description: num_heads=4, frl_config=60, framebuffer=8192M, max_resolution=7680x4320, max_instance=4
nvidia-359
Available instances: 0
Device API: vfio-pci
Name: GRID V100S-16Q
Description: num_heads=4, frl_config=60, framebuffer=16384M, max_resolution=7680x4320, max_instance=2

 

[dcsapak]

can you post the output of

cat /sys/bus/pci/devices/0000\:e1\:00.0/mdev_supported_types/nvidia-359/*

?

it looks like there is already a 'GRID V100S-8Q' instance (max_instance 4, but available 3)
since you cannot match & mix those type (AFAIK) if you already instanced such a type, you can only use that exact type again

 

[mohsenshahbazi]

so you say this limitation is make by GRID driver ?