Mellanox ConnectX-4 LX not getting 10GbE speeds

D

digipen79

New Member
Jan 30, 2025
18
2
3
I'm running PVE 9.1.1 on a Dell Poweredge R630, and I have a Dell/Mellanox ConnectX-4 LX installed, but on either connection I'm only getting 1GbE speed. I'm connected to Ubiquiti USW Aggregation with other devices connecting at 10GbE, so no issues with the switch. I am on firmware 14.27.6122 and am hoping I just need to upgrade, but nothing I can find has helped with getting that updated. Any thoughts on what I can try to get my full speeds on the Mellanox NIC?
 
How do you test the speed?
Which speeds did the NIC negotiate with the switch? ethtool <NIC> and look for the Speed: line.
 
Code: 
root@home1:~# ethtool eno1np0
Settings for eno1np0:
        Supported ports: [ Backplane ]
        Supported link modes:   1000baseKX/Full
                                10000baseKR/Full
                                25000baseCR/Full
                                25000baseKR/Full
                                25000baseSR/Full
        Supported pause frame use: Symmetric
        Supports auto-negotiation: Yes
        Supported FEC modes: None        RS      BASER
        Advertised link modes:  1000baseKX/Full
                                10000baseKR/Full
                                25000baseCR/Full
                                25000baseKR/Full
                                25000baseSR/Full
        Advertised pause frame use: Symmetric
        Advertised auto-negotiation: Yes
        Advertised FEC modes: None
        Speed: 1000Mb/s
        Lanes: 1
        Duplex: Full
        Auto-negotiation: on
        Port: Direct Attach Copper
        PHYAD: 0
        Transceiver: internal
        Supports Wake-on: g
        Wake-on: d
        Link detected: yes
 
Well, I did the simplest thing and went into the switch and forced it to 10GbE, and after a reboot of the server it is now at 10GbE. Easiest thing I could've done lol
 
well according to this, it negotiated to 1Gbit:
Speed: 1000Mb/s
Click to expand...

So it could also be a bad cable that caused the autonegotiation to scale back to the slower 1gbit. Keep an eye on how stable the connection is and how many retransmits you will see over time.

ip --statistics a will show how many packets are dropped or had errors.
 
I will keep an eye on it, something just happened after that that made my NAS share no longer connect to my server, I set it back to auto and it's staying at 10GbE but I cannot connect to my SMB share on my NAS, which is only a problem for 2 of my VMs. I can download the vbm and vbk files directly from the NAS and hopefully restore the disks to the server, but we'll see
 
It turned out to be my own fault, I had installed a Mellanox firmware updater during my troubleshooting, and something about that caused a disconnect upon reboot. Uninstalling and rebooting fixed it for me.
 
You must log in or register to reply here.
Top Bottom