Can't get Broadcom BCM57414 NetXtreme-E 10Gb/25Gb Nics to work at 10Gb/s

J

jsimhco

New Member
Jul 30, 2025
4
1
3
Can't get Broadcom BCM57414 NetXtreme-E 10Gb/25Gb Nics to work at 10Gb/s, or anything over 1Gb/s. Running in a Dell PowerEdge R640 connected to a Dell S5224F-ON switch. I've installed drivers per Broadcom's instructions. I've toggled MTU settings. Same cards work in our VMware environment using the same model switches on same model of servers. Any help is appreciated.https://techdocs.broadcom.com/us/en...ce-drivers-using-the-automated-installer.html

ethtool ens1f1np1
Settings for ens1f1np1:
Supported ports: [ FIBRE ]
Supported link modes: 1000baseX/Full
10000baseCR/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: RS BASER
Advertised link modes: 1000baseX/Full
10000baseCR/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Speed: 10000Mb/s
Lanes: 1
Duplex: Full
Auto-negotiation: on
Port: Direct Attach Copper
PHYAD: 1
Transceiver: internal
Supports Wake-on: d
Wake-on: d
Current message level: 0x000020c1 (8385)
drv rx_err tx_err hw
Link detected: yes
 
I've also tried creating a bridge with the 10Gb ports, same result. Bond with 10Gb ports that is on bridge, same result. Moved gateway from original bridge to 10Gb bridge, same issue. Iperf only showing 1Gb/s to host or vm on host using the 10Gb bridge.
 
Just a wild guess but maybe your transceivers / DAC cables are not fully compatible and allow negotiation only with 1Gb?
If you have the possibility test with Cisco compatible transceivers / DAC cables, they often work with most gear.

And upgrade the NIC firmware if you have not done that already, often solves some issues.
 
Last edited:
MarkusKo said:
Just a wild guess but maybe your transceivers / DAC cables are not fully compatible and allow negotiation only with 1Gb?
If you have the possibility test with Cisco compatible transceivers / DAC cables, they often work with most gear.

And upgrade the NIC firmware if you have not done that already, often solves some issues.
Click to expand...
Using DELL EMC DAC-SFP-10G-5M. What cisco compatible transceivers / DAC cables would you recommend?
 
Can't really recommend anything because i don't have enterprise class gear but you could always try fs.com DAC cables, they are quite affordable.

If you have more optical gear and budget you could get some programmable DAC's or transceivers from flexoptics or fs.com. You could program one end for dell and the other for broadcom. There are other alternatives out there but i would test with a fs.com DAC / Cisco compatible first - not much lost if it does not work ...
 
Putting this here so that anyone who has a similar issue can replicate.

What i ended up having to do is install the drivers for the nics via this: https://techdocs.broadcom.com/us/en...ce-drivers-using-the-automated-installer.html

cd bcm_234.1.124.0b/utils/linux_installer/
bash install.sh -i 3B:00.0 -2

and do that for each nic without cables/DAC/transceivers connected.

Then i moved the gateway and IP config form the original 1Gb/s bridge to the 10Gb/s bridge i have configured to use a bond of the 2 10Gb/s nics.

After that iperf showed speeds that you would expect from this kind of upgrade.

Use case: NFS storage for vms/lxcs with Proxmox 8.0 over 10Gb/s connections.
 
  • Like
Reactions: MarkusKo
You must log in or register to reply here.
Top Bottom