Mellanox Connect-X 3 Pro and 40Gbit

[kira12]

Hello Guys,

i use a Dual Connect-X 3 Pro and one Port with SFP+ 10G works fine. But the 2. Port with an DAC Cable with 40Gbit to the 2. Computer with exactly the Same Hardware and Software dont bring the Port up and running. With Freebsd is the Nic and the Cable running. What can i do to solve the Problem?
I use 6.2-15 under Debian 10.3.
Port 1 and 2:
ens5f0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 3c:a8:2a:fd:ae:00 txqueuelen 1000 (Ethernet)
RX packets 28506707 bytes 7803177988 (7.2 GiB)
RX errors 0 dropped 33070 overruns 0 frame 0
TX packets 22943522 bytes 5871778544 (5.4 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

ens5f1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 3c:a8:2a:fd:ae:08 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

best regards ré

 

[wolfgang]

Hi,

did you upgrade the NIC firmware?
Mellanox NICs are known for incompatible with new kernels and old controller firmware.

 

[kira12]

Hi,

thanks for the responce. I use the 2. latest Firmware from HPE. I use an Infiniband DAC Cable and both CArds have an IP in the same Subnet. But i cant Ping. Do i need somthing more?

best regards René

Update: the ibstatus show both cards in 56G Mode but i the Quicspecs say they only support 40G. Status is also not active. Its initialised

 

[wolfgang]

Update: the ibstatus show both cards in 56G Mode but i the Quicspecs say they only support 40G. Status is also not active. Its initialised

If I remember correctly you need an extra key and Mellanox switch to upgrade to 56G

 

[kira12]

Hi,

i only had to set the Port Type to eth via mlxconfig. That was the Goal, thanks for the Help.

best regards