Installing Mellanox/Nvidia drivers on Proxmox VE (7.2)

[Veidit]

Good afternoon

I have fine Mellanox/Nvidia ConnectX-5 EN card running firmware version: 16.33.1048
root@kg-virt01:~# mlxconfig -d /dev/mst/mt4119_pciconf0 query

Device #1:
----------

Device type: ConnectX5
Name: MCX516A-CCA_Ax
Description: ConnectX-5 EN network interface card; 100GbE dual-port QSFP28; PCIe3.0 x16; tall bracket; ROHS R6
Device: /dev/mst/mt4119_pciconf0

My question is if and if so how I should install the drivers from Mellanox/Nvidia.
For storage right now I am using NFS but I hope to in the future run NFS over ROCE(v2).
Looking at the latest driver (either the full package or only the ethernet drivers) I can see that the latest version is 5.6-2.0.9.0 and that only supports Debian 11.2
Proxmox VE 7.2 is based on Debian 11.3 and it seems to take it's time before new drivers are released.
What is recommended here, should I use the driver that came with the OS or is the some way to install the drivers from Nvidia/Mellanox?

 

[mira]

Actually, the kernel PVE 7 uses is based on Ubuntu's with some additional patches on top.
So for PVE 7.2 this would be kernel 5.15 based on the one in Ubuntu 22.04. If Mellanox provides a driver for Ubuntu 22.04, you might be in luck.

Is there no support in the kernel for that card?

 

[Veidit]

Actually, the kernel PVE 7 uses is based on Ubuntu's with some additional patches on top.
So for PVE 7.2 this would be kernel 5.15 based on the one in Ubuntu 22.04. If Mellanox provides a driver for Ubuntu 22.04, you might be in luck.

Is there no support in the kernel for that card?

Thanks.

I didn't know about that, I will give the Ubuntu driver a try!

There is support for the card in the kernel but what I can see people recommend to use Mellanox/Nvidias driver.

My network isn't fully set up yet so it might take me a few days to get the Ubuntu driver tested.

 

[mira]

To be honest, if the in-tree driver works for you I wouldn't bother with the out-of-tree one.
Keeping that up-to-date and getting everything to work again on every update might be lots of work for possibly little to no gain.

 

[Veidit]

To be honest, if the in-tree driver works for you I wouldn't bother with the out-of-tree one.
Keeping that up-to-date and getting everything to work again on every update might be lots of work for possibly little to no gain.

That is valid. I can install and upgrade the firmware as it is now and that's important. Thanks!