mlx5 sub function passhrough for VM (mdev)

auranext

Well-Known Member
Jun 5, 2018
54
2
48
123
Hello,

I have successfully tested a pve VXLAN-EVPN setup with pci passthrough and switchdev (mlx5 mode eswitch) .
However, the 127 VF limit is too restrictive. That's why I'm looking for another more scalable solution.
I noticed that mellanox offers in addition to the usual sr-iov devices (PF, VF) the SF device
it's a pci sub function that can be passthrough to a VM with the help of a mediated device (mdev)
So I am trying to do pci sub funtion passthrough, I have done:

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"
enable switchdev mode eswitch


the problem is that I can't find the mdev file structure in sysfs
directory mdev_supported_types does not exist...

/sys/bus/pci/devices/<mellanox-netdev-pci-addr>/mdev_supported_types

but the native mdev directory exists

/sys/bus/mdev/devices/

for the moment the mdev concept and usage are a bit confusing for me
I think I missed something...

some help would be appreciate

mellanox OOB driver and fw are up2date
kernel is 5.15.19.2
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!