install ConnectX-3 on ProxMox v5.x

FATeknollogee

Member
May 3, 2017
35
2
13
58
Is it possible to install drivers for Mellanox ConnectX-3 in Proxmox v5.x?
The Mellanox site only has drivers for Debian 8.3.
I tried using the 8.3 drivers but I'm getting some errors.

# ./install --distro debian8.3
Log: /tmp/ofed.build.log
Logs dir: /tmp/mlnx-en.31026.logs

Below is the list of mlnx-en packages that you have chosen
(some may have been added by the installer due to package dependencies):

ofed-scripts
mlnx-en-utils
mlnx-en-dkms
mstflint

This program will install the mlnx-en package on your machine.
Note that all other Mellanox, OEM, OFED, RDMA or Distribution IB packages will be removed.
Those packages are removed due to conflicts with mlnx-en, do not reinstall them.

Do you want to continue?[y/N]:y

Checking SW Requirements...
One or more required packages for installing mlnx-en are missing.
/lib/modules/4.10.1-2-pve/build/scripts is required for the Installation.
Attempting to install the following missing packages:
m4 autoconf swig automake dkms autotools-dev python-libxml2 graphviz debhelper chrpath dpatch quilt libltdl-dev linux-headers-4.10.1-2-pve
Failed command: apt-get install -y m4 autoconf swig automake dkms autotools-dev python-libxml2 graphviz debhelper chrpath dpatch quilt libltdl-dev linux-headers-4.10.1-2-pve
 

manu

Proxmox Staff Member
Mar 3, 2015
806
66
28
since the kernel we provide is quite recent, you usually should not need to rebuild out of tree kernel drivers

using the current kernel, is your mellanox adapter detected with the proper driver ?
you can inspect with lspci -k the content of your PCI bus, and see is a kernel was attached to your hardware
 

FATeknollogee

Member
May 3, 2017
35
2
13
58
The Mellanox adapter is there, but it does not show up in the GUI under Network tab because by default it is set to IB (not Ethernet)
root@proxmox1-la2:~# cat /sys/bus/pci/devices/0000\:82\:00.0/mlx4_port1
auto(ib)
root@proxmox1-la2:~# cat /sys/bus/pci/devices/0000\:82\:00.0/mlx4_port2
auto(ib)

I don't know of a clean way to change it from IB to Ethernet without installing the drivers
 

FATeknollogee

Member
May 3, 2017
35
2
13
58
82:00.0 Network controller: Mellanox Technologies MT27500 Family [ConnectX-3]
Subsystem: Mellanox Technologies MT27500 Family [ConnectX-3]
Kernel driver in use: mlx4_core
Kernel modules: mlx4_core
 

manu

Proxmox Staff Member
Mar 3, 2015
806
66
28
what is the output of the lspci -k command ?
 

FATeknollogee

Member
May 3, 2017
35
2
13
58
what is the output of the lspci -k command ?

82:00.0 Network controller: Mellanox Technologies MT27500 Family [ConnectX-3]
Subsystem: Mellanox Technologies MT27500 Family [ConnectX-3]
Kernel driver in use: mlx4_core
Kernel modules: mlx4_core
 

manu

Proxmox Staff Member
Mar 3, 2015
806
66
28
ok
so you have a working driver with this adapter, and you should be able to use it normally

now I don't have Mellanox hardware in my lab, refer to your vendor on how to enable the Ethernet Mode you mean if such a thing is possible

if the Linux instructions Mellanox give you are not working, please report here
 

fabian

Proxmox Staff Member
Staff member
Jan 7, 2016
7,609
1,431
164

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!