Unable to reach 10Gbps

D

DominicDeX

New Member
May 3, 2024
3
0
1
Update - 20 June 2024.

I managed to somewhat resolve this issue.

I booted my windows PC to Ubuntu and did iperf there. I got 10Gbs up and down. So I guess its an issue with Windows.


Hi all,

I was recently introduced to Proxmox VE by a close friend of mine and I have been playing around alot for the past few weeks. However, I have been running into some issues with my network speed.

Currently, I have 2 setups:

1st setup - Main Desktop
Z790 Gigabyte Aorus Master X on i9-14900K
Marvel AQtion v 3.1.6.0

2nd setup - NAS Server
Proxmox VE
H110M on i5-7400

I just recently purchased a 10G NIC card, AQC113C NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] and I want to set up a 10Gbps internal network. I have also just bought a 10G network switch and both my main desktop and NAS are connected to it.

ip link on my NAS Server (Proxmox Host console):
1718300393654.png

ethtool enp1s0:
1718300456038.png

Based on my knowledge, (what little I have so far), if both my LAN ports are connected via 10G with CAT 8 cables, I should have both 10Gbps transfer speeds. However, after testing iperf on my network, here is what I have seen.

Main Desktop 192.168.2.32/24
Proxmox host 192.168.2.222/24

iperf3 -c 192.168.2.32
1718300641601.png
From Promox to the Main desktop, I am getting 2.5gbps of transfer speed.

iperf -c 192.168.2.222
1718300725670.png
From my Desktop to Proxmox, around 10Gbps of transfer speed.

/etc/network/interfaces
1718300920648.png

Moreover, my switch shows that both my desktop and proxmox server are connected at 10G auto duplex. I have checked multiple times to ensure it has 10G connectivity and I have tried to set the MTU to 9000, all to no avail.

Currently, I am scratching my head and figuring out why am I receiving this type of speed. Any kind soul can kindly help me before my GF decides to kick me to the curb?

Much appreciated!
 
Last edited:
Hi! I have the same problem. Proxmox 8.2.4 and ASUS with Aquantia AQC113CS chip.

Did you manage to get it working or what was your fix?

I have found some reasons but no solution yet. Getting 1000/2500 speeds on a 10G card. Everything is reporting link as 10G and cables are good.

From Aquantia driver readme file, seems like the AQC113CS chip does not like to be bonded or bridged:
WARNING:
AQtion driver is built with the LRO (Large Receive Offload) feature enabled by default.
This option offers the lowest CPU utilization for receives, but it is completely
incompatible with *routing/ip forwarding* and *bridging*.
If you need ip forwarding or bridging, please make sure to disable LRO using the
compile time options described in the LRO section below.
NB! If LRO is enabled, attempts to use ip forwarding or bridging can result in
low throughput or even a kernel panic.
Click to expand...

To turn the LRO stuff off, I found that someone got help by adding to the etc/network/interfaces file rows "offload-..." under the 10G NIC config.
e.g:

Code: 
iface eth0 inet static
    address 10.0.3.1/255.255.248.0
    gateway 10.0.2.10
    offload-tx off
    offload-sg off
    offload-tso off
    offload-gso off
    offload-lro off

This seemed to help a little, but still not 10G speeds.
 
Hi @vanarebane, what does your setup look like?

My setup didn't have any issues per say. It was because I was using Windows iperf to my Proxmox iperf. I read somewhere that iperf on windows isn't 100% accurate.

So in order to test this theory, I had my windows desktop booted to Ubuntu and ran my iperf there. I have 9.1Gbps up and down.
 
My setup: unRaid (that runs in VM) > Proxmox (VM host with ASUS XG-C100C v2) > RT-BE88U (router) > Fiber to MikroTik CRS305-1G-4S+IN > PC with Mellanox Connectx-3.
Or by going with MTU settings: 9000 > 9000 > 1500 (router 10G RJ45) > 1500 (router 10G SFP+) > 9000 > 9000
But as I understand, having MTU 1500 does not mean speeds are limited to 1G or 2.5G.

After the LTO settings and with using iperf3 I now get 1G/9G speed in one direction though, from unRaid > PC.
But between Proxmox > PC I get 2.5G/2.5G speeds.
And between Proxmox > RT-BE88U I get 1G/2.5G

I'm using iperf3 with multithreads, to get the most out of the cable, but it seems either to be limited.

I'll write again to ASUS and see what they reply.
 
skip the cheap router and plug directly pc to pc. And best, test with win10 on both with latest driver. if you do get full performance, then check the code of the linux driver. start by having a confirm working system, then adapt.
 
Did this ever get resolved? I just picked up a TRX50 board that has the Marvell AQC113 chip on it. I am really tight on space for PCIe devices and don't know if I can manage adding a PCIe NIC.
 
Its very strange with this AQtion AQC113 nic

I have a Ugreen DXP4800Plus where I replaced the stupid UGOS with Proxmox. Without any special doings I get the full 10Gb:

Code: 
root@pve:~# iperf3 -c 192.168.55.10
Connecting to host 192.168.55.10, port 5201
[  5] local 192.168.55.5 port 43338 connected to 192.168.55.10 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  1.10 GBytes  9.43 Gbits/sec    0   1.66 MBytes      
[  5]   1.00-2.00   sec  1.10 GBytes  9.42 Gbits/sec    0   1.83 MBytes      
[  5]   2.00-3.00   sec  1.10 GBytes  9.42 Gbits/sec    0   1.83 MBytes      
[  5]   3.00-4.00   sec  1.09 GBytes  9.41 Gbits/sec    0   1.92 MBytes      
[  5]   4.00-5.00   sec  1.10 GBytes  9.42 Gbits/sec    0   2.02 MBytes      
[  5]   5.00-6.00   sec  1.10 GBytes  9.42 Gbits/sec    0   2.02 MBytes      
[  5]   6.00-7.00   sec  1.10 GBytes  9.42 Gbits/sec    0   2.02 MBytes      
[  5]   7.00-8.00   sec  1.10 GBytes  9.42 Gbits/sec    0   2.02 MBytes      
[  5]   8.00-9.00   sec  1.10 GBytes  9.41 Gbits/sec    0   2.02 MBytes      
[  5]   9.00-10.00  sec  1.09 GBytes  9.41 Gbits/sec    0   2.02 MBytes      
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  11.0 GBytes  9.42 Gbits/sec    0             sender
[  5]   0.00-10.04  sec  11.0 GBytes  9.38 Gbits/sec                  receiver

This NAS has 2 Network ports, but I am using the 10Gb only. Everything is working very, very fine at this NAS (I just pass the whole SATA Controller into a VM, the OS and other VMs are running from nvme).

Then I thought it could be a good idea to add the Ugreen DXP480T Plus into my homelab and get rid of some other pve-hosts.
I found it is using the same Action AQC113 network chip:

Code: 
root@pve:~# lspci -nn | grep -i aquantia
03:00.0 Ethernet controller [0200]: Aquantia Corp. AQtion AQC113 NBase-T/IEEE 802.3an Ethernet Controller [Antigua 10G] [1d6a:04c0] (rev 03)

But - I do not get 10 Gbit, iperf3 shows only:

Code: 
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 10.0.0.58, port 53770
[  5] local 192.168.55.10 port 5201 connected to 10.0.0.58 port 53786
[ ID] Interval           Transfer     Bitrate         Rwnd
[  5]   0.00-1.00   sec  90.9 MBytes   763 Mbits/sec   705 KBytes                
[  5]   1.00-2.00   sec  97.8 MBytes   821 Mbits/sec   715 KBytes                
[  5]   2.00-3.00   sec  82.1 MBytes   685 Mbits/sec   715 KBytes                
[  5]   3.00-4.00   sec  89.3 MBytes   753 Mbits/sec   715 KBytes                
[  5]   4.00-5.00   sec   100 MBytes   840 Mbits/sec   715 KBytes                
[  5]   5.00-6.00   sec   100 MBytes   842 Mbits/sec   715 KBytes                
[  5]   6.00-7.00   sec   101 MBytes   845 Mbits/sec   701 KBytes                
[  5]   7.00-8.00   sec   101 MBytes   843 Mbits/sec   715 KBytes                
[  5]   8.00-9.00   sec   100 MBytes   841 Mbits/sec   701 KBytes                
[  5]   9.00-10.00  sec  98.9 MBytes   830 Mbits/sec   715 KBytes                
[  5]  10.00-10.04  sec  3.59 MBytes   845 Mbits/sec   696 KBytes                
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.04  sec   965 MBytes   806 Mbits/sec

Very disappointing. Both PVE hosts are up2date (community only), the linkspeed shows on both 10Gbit and they are attached to the same Unifi-switch.
For sure I switched the ports and the cables and and and - with no changes in speed - so it has to be a driver problem on the pve-host.

I searched the web a lot but without finding a solution. I also tried to compile an fw-update for the nix but got to much errors while make - mostly because the used kernel was not right (it's 6.8.12-7-pve). I also tried an older Kernel (6.5) but with no success.

I will try now a clean reinstall of the system with some other operation systems (Debian 12, 11, ubuntu, redhat, windows ...)
 
Last edited:
You must log in or register to reply here.
Top Bottom