Hi
I am working with proxmox hypervisors since a few years, 5.3 , 6.2 and now 7.x
considering that we virtualize routers we are always looking for better packet processing performance.
After some reseach we decide to focus on hardware acceleration solution based on mellanox connectx-5 and 6
For this purpose we plugged a connectx-6 card into a HPE DL360Gen10 server (16 cores, 768GB ram), upgrade kernel to pve-kernel-5.15.x and compiling MLNX-OFED 5.x
First we have implemented SRIOV with representor interface on a PVE7 node (eswitch mod)
It is easy to deploy and very performant but non hotplug ready and non scalable, 256 VF max (mellanox lies in technical specs of connectx cards)
however if for someone the constraints are acceptable it is very functional and offload with tc works very well (zero interrupts on hypervisor !)
Another important constaint is the guest OS need to support mlx5 driver.
Because of those contraints now we are exploring an other setup based on VDPA and mlnx scalable function
For the moment I have realized a POC I can say it works, guest OS have virtio driver, it is totally transparent.
But it is very early to say more...
smarttubenext because I encounter kernel stability issues and the documentations or howto are very poor or simply inexistant
because my time is limited and the packet processing demand increases as time passes I am looking for a partner to complete this work
the subject is vast and very experimental, I will post when I will have more solid information
contact me in private if you want to discuss it
++
