Slow bond with Realtek quad NICs

[lholota]

I have recently moved my homelab from enterprise hardware to consumer one due to power consumption and despite the set up being very similar I'm getting really poor network performance.

My current set up

• 2 Proxmox nodes - each with Ryzen 5700, 32GB RAM, NVMe as system disk and quad NICs (RTL8111/8168/8411 PCIe x1)
• Services network - 2 ports on each nodes are bonded (LACP 802.3ad, hash policy layer2+3) - this is traffic from clients
• Migration network - 2 ports on each nodes are bonded (LACP 802.3ad, hash policy layer2) - this is dedicated to traffic between the nodes
• Both nodes are running up to date proxmox with kernel 6.8.8-1-pve

Switch is Ubiquiti US-24 with LAGs defined (there's really no other configuration specific to LAG in Unifi)

When I use iperf to measure the performance the max speed I get is 940-960 Mbits which is less than one NIC alone. I had previously exactly the same set up on two Dell R530s using the integrated LOMs and I was getting around 1600-1700 Mbits in iperf.

I have read about problems with drivers for these Realtek NICs but the connection seems stable, it's been running fine for about a month but the speed has been this poor from the beginning. Any help is highly appreciated!

 

[shanreich]

Those are 1 GBit NICS, right? For one connection LACP at most has the speed of one port, so 940/960 MBit is exactly what you would expect (almost 1G).

 

[lholota]

Those are 1 GBit NICS, right? For one connection LACP at most has the speed of one port, so 940/960 MBit is exactly what you would expect (almost 1G).

Yeah, they are 1 Gbps NICs but the R530 also had them although with a different chip and iperf was able to utilize both of them and report speed of approx. 1,6 Gbps that's why I'm confused the experience changed even though the hardware has similar specs