Throughput issues with recently installed Gig Ethernet card - Dell PowerEdge T320, Proxmox/Debian, Broadcom NetXtreme BCM5722

jsalas424

Active Member
Jul 5, 2020
143
3
38
34
Hi all,

I recently added a node to a Proxmox cluster and was setting up a dedicated server <-> server ethernet connection with some spare NICs but I am getting significant throughput issues on the new card.

I am suspicious of the NIC I installed in the T320 and NOT of the NIC in the node because the node's NIC is integrated to the motherboard.

I installed a Broadcom Limited NetXtreme BCM5722 Gigabit Ethernet in a PCIe slot and ran iperf3 tests against that connection. I'm getting ~ 1/10th of what I should be getting.

Code:
root@Server:~# iperf3 -s -f K
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 10.0.0.1, port 55302
[  5] local 10.0.0.2 port 5201 connected to 10.0.0.1 port 55304
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  2.68 MBytes  2745 KBytes/sec                 
[  5]   1.00-2.00   sec  2.54 MBytes  2598 KBytes/sec                 
[  5]   2.00-3.00   sec  2.67 MBytes  2733 KBytes/sec                 
[  5]   3.00-4.00   sec  2.72 MBytes  2790 KBytes/sec                 
[  5]   4.00-5.00   sec  2.57 MBytes  2633 KBytes/sec                 
[  5]   5.00-6.00   sec  2.57 MBytes  2629 KBytes/sec                 
[  5]   6.00-7.00   sec  2.71 MBytes  2780 KBytes/sec                 
[  5]   7.00-8.00   sec  2.72 MBytes  2786 KBytes/sec                 
[  5]   8.00-9.00   sec  2.64 MBytes  2705 KBytes/sec                 
[  5]   9.00-10.00  sec  2.75 MBytes  2813 KBytes/sec                 
[  5]  10.00-10.01  sec  9.90 KBytes  1131 KBytes/sec                 
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.01  sec  26.6 MBytes  2720 KBytes/sec                  receiver

Do I need to install drivers for this new card? How?

There are 2 x Broadcom Limited NetXtreme BCM5720 Gigabit Ethernet PCIe already in the server and working perfectly fine. Here's an iperf3 test of the other NICs:

Code:
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.1.24, port 41744
[  5] local 192.168.1.129 port 5201 connected to 192.168.1.24 port 41746
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   110 MBytes  112945 KBytes/sec                 
[  5]   1.00-2.00   sec   111 MBytes  113185 KBytes/sec                 
[  5]   2.00-3.00   sec   111 MBytes  113171 KBytes/sec                 
[  5]   3.00-4.00   sec   110 MBytes  113107 KBytes/sec                 
[  5]   4.00-5.00   sec   111 MBytes  113154 KBytes/sec                 
[  5]   5.00-6.00   sec   111 MBytes  113193 KBytes/sec                 
[  5]   6.00-7.00   sec   110 MBytes  113137 KBytes/sec                 
[  5]   7.00-8.00   sec   110 MBytes  113026 KBytes/sec                 
[  5]   8.00-9.00   sec   110 MBytes  113006 KBytes/sec                 
[  5]   9.00-10.00  sec   111 MBytes  113192 KBytes/sec                 
[  5]  10.00-10.00  sec   276 KBytes  111679 KBytes/sec                 
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.00  sec  1.08 GBytes  113111 KBytes/sec                  receiver
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------

What am I doing wrong here?
 
Here's the network config on both Proxmox hosts with "Cluster Network" corresponding to the troublemaker
1623629320570.png
1623629331562.png