nics show as 10g, but iperf3 shows 1g transfer speeds

[lknite]

I have three hosts, each with 2 10gb nics. I've been using 1 port for storage on each host.

I discovered that all three of the storage nics were actually running at 1gb, and seem to have fixed this using the command:
ethtool -s enp1s0f1 autoneg on speed 10000 duplex full

After running that command I can see the nics are showing Speed 10000 Mb/s:

root@pve-b:~# ethtool enp16s0f1
Settings for enp16s0f1:
        Supported ports: [ TP ]
        Supported link modes:   100baseT/Full
                                1000baseT/Full
                                10000baseT/Full
        Supported pause frame use: Symmetric
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10000baseT/Full
        Advertised pause frame use: Symmetric
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Speed: 10000Mb/s
        Duplex: Full
        Auto-negotiation: on
        Port: Twisted Pair
        PHYAD: 0
        Transceiver: internal
        MDI-X: Unknown
        Supports Wake-on: d
        Wake-on: d
        Current message level: 0x00000007 (7)
                               drv probe link
        Link detected: yes

However, the transfer speed still seems to be running at 1gb for some reason:

root@pve-b:~# iperf3 -c 10.0.0.23
Connecting to host 10.0.0.23, port 5201
[  5] local 10.0.0.22 port 41866 connected to 10.0.0.23 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   113 MBytes   948 Mbits/sec    6    423 KBytes    
[  5]   1.00-2.00   sec   111 MBytes   930 Mbits/sec    0    486 KBytes    
[  5]   2.00-3.00   sec   106 MBytes   889 Mbits/sec    1    498 KBytes

CAT6 is in use and all cables are 6ft long. Switch is definitely 10gb on each port. Verified all three hosts show nic at 10000Mb/s. All hosts tested between each other with same result.

Is there something that needs to be restarted before the 10gb speed will be used?

 

[lknite]

Nevermind, I was literally testing the wrong port via the ip:

root@pve-b:~# iperf3 -c 10.0.1.23
Connecting to host 10.0.1.23, port 5201
[  5] local 10.0.1.22 port 57654 connected to 10.0.1.23 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  1.09 GBytes  9.37 Gbits/sec   38   1.32 MBytes       
[  5]   1.00-2.00   sec  1.09 GBytes  9.32 Gbits/sec    0   1.83 MBytes       
[  5]   2.00-3.00   sec  1.09 GBytes  9.35 Gbits/sec    0   1.99 MBytes       
[  5]   3.00-4.00   sec  1.08 GBytes  9.29 Gbits/sec    0   2.00 MBytes       
[  5]   4.00-5.00   sec  1.08 GBytes  9.30 Gbits/sec    0   2.00 MBytes       
[  5]   5.00-6.00   sec  1.09 GBytes  9.34 Gbits/sec    0   2.01 MBytes       
[  5]   6.00-7.00   sec  1.08 GBytes  9.31 Gbits/sec    0   2.01 MBytes       
[  5]   7.00-8.00   sec  1.09 GBytes  9.40 Gbits/sec    0   2.01 MBytes       
[  5]   8.00-9.00   sec  1.09 GBytes  9.37 Gbits/sec    0   2.01 MBytes       
[  5]   9.00-10.00  sec  1.09 GBytes  9.37 Gbits/sec    0   2.01 MBytes