Throughput issues with recently installed Gig Ethernet card - Dell PowerEdge T320, Proxmox/Debian, Broadcom NetXtreme BCM5722

jsalas424

Member
Jul 5, 2020
141
2
23
34
Hi all,

I recently added a node to a Proxmox cluster and was setting up a dedicated server <-> server ethernet connection with some spare NICs but I am getting significant throughput issues on the new card.

I am suspicious of the NIC I installed in the T320 and NOT of the NIC in the node because the node's NIC is integrated to the motherboard.

I installed a Broadcom Limited NetXtreme BCM5722 Gigabit Ethernet in a PCIe slot and ran iperf3 tests against that connection. I'm getting ~ 1/10th of what I should be getting.

Code:
root@Server:~# iperf3 -s -f K
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 10.0.0.1, port 55302
[  5] local 10.0.0.2 port 5201 connected to 10.0.0.1 port 55304
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  2.68 MBytes  2745 KBytes/sec                 
[  5]   1.00-2.00   sec  2.54 MBytes  2598 KBytes/sec                 
[  5]   2.00-3.00   sec  2.67 MBytes  2733 KBytes/sec                 
[  5]   3.00-4.00   sec  2.72 MBytes  2790 KBytes/sec                 
[  5]   4.00-5.00   sec  2.57 MBytes  2633 KBytes/sec                 
[  5]   5.00-6.00   sec  2.57 MBytes  2629 KBytes/sec                 
[  5]   6.00-7.00   sec  2.71 MBytes  2780 KBytes/sec                 
[  5]   7.00-8.00   sec  2.72 MBytes  2786 KBytes/sec                 
[  5]   8.00-9.00   sec  2.64 MBytes  2705 KBytes/sec                 
[  5]   9.00-10.00  sec  2.75 MBytes  2813 KBytes/sec                 
[  5]  10.00-10.01  sec  9.90 KBytes  1131 KBytes/sec                 
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.01  sec  26.6 MBytes  2720 KBytes/sec                  receiver

Do I need to install drivers for this new card? How?

There are 2 x Broadcom Limited NetXtreme BCM5720 Gigabit Ethernet PCIe already in the server and working perfectly fine. Here's an iperf3 test of the other NICs:

Code:
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.1.24, port 41744
[  5] local 192.168.1.129 port 5201 connected to 192.168.1.24 port 41746
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   110 MBytes  112945 KBytes/sec                 
[  5]   1.00-2.00   sec   111 MBytes  113185 KBytes/sec                 
[  5]   2.00-3.00   sec   111 MBytes  113171 KBytes/sec                 
[  5]   3.00-4.00   sec   110 MBytes  113107 KBytes/sec                 
[  5]   4.00-5.00   sec   111 MBytes  113154 KBytes/sec                 
[  5]   5.00-6.00   sec   111 MBytes  113193 KBytes/sec                 
[  5]   6.00-7.00   sec   110 MBytes  113137 KBytes/sec                 
[  5]   7.00-8.00   sec   110 MBytes  113026 KBytes/sec                 
[  5]   8.00-9.00   sec   110 MBytes  113006 KBytes/sec                 
[  5]   9.00-10.00  sec   111 MBytes  113192 KBytes/sec                 
[  5]  10.00-10.00  sec   276 KBytes  111679 KBytes/sec                 
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.00  sec  1.08 GBytes  113111 KBytes/sec                  receiver
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------

What am I doing wrong here?
 
Here's the network config on both Proxmox hosts with "Cluster Network" corresponding to the troublemaker
1623629320570.png
1623629331562.png
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!