10Gb card and issues

[cvega]

Hi chaps

I've added a 10Gbe card to my proxmox machine, as well as one to my xigmanas. The network config is as follows:

NAS
LAGG0 (teamed 2 x 1gbit), 192.168.0.100
OPT_1 (10gb card) 192.168.20.1/24, no gateway

Proxmox
enp6s0 -> vmbr0 192.168.0.15/24, gateway 192.168.0.1
ens6f0 (10gb card interface 0) 192.168.20.2/24, no gateway
ens6f1 (10gb card interface 1, inactive)

At the moment, traffic only seems to flow one way, when I test with iperf3.

When xigmanas is the listener, we get no traffic in:

NAS side

xigmanas: ~# iperf3 -s -B 192.168.20.1
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.20.2, port 36560
[ 5] local 192.168.20.1 port 5201 connected to 192.168.20.2 port 36562
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 0.00 Bytes 0.00 bits/sec
[ 5] 1.00-2.00 sec 0.00 Bytes 0.00 bits/sec
[ 5] 2.00-3.00 sec 0.00 Bytes 0.00 bits/sec
[ 5] 3.00-4.00 sec 0.00 Bytes 0.00 bits/sec
[ 5] 4.00-5.00 sec 0.00 Bytes 0.00 bits/sec
[ 5] 5.00-6.00 sec 0.00 Bytes 0.00 bits/sec
[ 5] 6.00-7.00 sec 0.00 Bytes 0.00 bits/sec
[ 5] 7.00-8.00 sec 0.00 Bytes 0.00 bits/sec
[ 5] 8.00-9.00 sec 0.00 Bytes 0.00 bits/sec
[ 5] 9.00-10.00 sec 0.00 Bytes 0.00 bits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.00 sec 0.00 Bytes 0.00 bits/sec receiver


proxmox server:
root@zenon:~# iperf3 -c 192.168.20.1
Connecting to host 192.168.20.1, port 5201
[ 5] local 192.168.20.2 port 36562 connected to 192.168.20.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 358 KBytes 2.93 Mbits/sec 2 8.74 KBytes
[ 5] 1.00-2.00 sec 0.00 Bytes 0.00 bits/sec 1 8.74 KBytes
[ 5] 2.00-3.00 sec 0.00 Bytes 0.00 bits/sec 0 8.74 KBytes
[ 5] 3.00-4.00 sec 0.00 Bytes 0.00 bits/sec 1 8.74 KBytes
[ 5] 4.00-5.00 sec 0.00 Bytes 0.00 bits/sec 0 8.74 KBytes
[ 5] 5.00-6.00 sec 0.00 Bytes 0.00 bits/sec 0 8.74 KBytes
[ 5] 6.00-7.00 sec 0.00 Bytes 0.00 bits/sec 1 8.74 KBytes
[ 5] 7.00-8.00 sec 0.00 Bytes 0.00 bits/sec 0 8.74 KBytes
[ 5] 8.00-9.00 sec 0.00 Bytes 0.00 bits/sec 0 8.74 KBytes
[ 5] 9.00-10.00 sec 0.00 Bytes 0.00 bits/sec 0 8.74 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 358 KBytes 293 Kbits/sec 5 sender
[ 5] 0.00-10.00 sec 0.00 Bytes 0.00 bits/sec receiver




When I reverse the roles, this happens:

NAS side:
xigmanas: ~# iperf3 -c 192.168.20.2
#Connecting to host 192.168.20.2, port 5201
[ 5] local 192.168.20.1 port 15643 connected to 192.168.20.2 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 1.04 GBytes 8.94 Gbits/sec 0 3.01 MBytes
[ 5] 1.00-2.00 sec 1.03 GBytes 8.88 Gbits/sec 156 2.93 MBytes
[ 5] 2.00-3.00 sec 1.03 GBytes 8.89 Gbits/sec 0 3.00 MBytes
[ 5] 3.00-4.00 sec 1.03 GBytes 8.89 Gbits/sec 68 2.22 MBytes
[ 5] 4.00-5.00 sec 1.03 GBytes 8.88 Gbits/sec 102 2.31 MBytes
[ 5] 5.00-6.00 sec 1.04 GBytes 8.89 Gbits/sec 0 3.00 MBytes
[ 5] 6.00-7.00 sec 1.03 GBytes 8.88 Gbits/sec 105 2.47 MBytes
[ 5] 7.00-8.00 sec 1.03 GBytes 8.89 Gbits/sec 0 3.00 MBytes
[ 5] 8.00-9.00 sec 1.03 GBytes 8.89 Gbits/sec 0 3.00 MBytes
[ 5] 9.00-10.00 sec 1.03 GBytes 8.89 Gbits/sec 0 3.00 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 10.4 GBytes 8.89 Gbits/sec 431 sender
[ 5] 0.00-10.11 sec 10.4 GBytes 8.79 Gbits/sec receiver


Proxmox host:
root@zenon:~# iperf3 -s -B 192.168.20.2
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.20.1, port 31203
[ 5] local 192.168.20.2 port 5201 connected to 192.168.20.1 port 15643
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 945 MBytes 7.93 Gbits/sec
[ 5] 1.00-2.00 sec 1.03 GBytes 8.88 Gbits/sec
[ 5] 2.00-3.00 sec 1.03 GBytes 8.89 Gbits/sec
[ 5] 3.00-4.00 sec 1.03 GBytes 8.89 Gbits/sec
[ 5] 4.00-5.00 sec 1.03 GBytes 8.88 Gbits/sec
[ 5] 5.00-6.00 sec 1.04 GBytes 8.89 Gbits/sec
[ 5] 6.00-7.00 sec 1.03 GBytes 8.88 Gbits/sec
[ 5] 7.00-8.00 sec 1.03 GBytes 8.89 Gbits/sec
[ 5] 8.00-9.00 sec 1.03 GBytes 8.89 Gbits/sec
[ 5] 9.00-10.00 sec 1.03 GBytes 8.89 Gbits/sec
[ 5] 10.00-10.11 sec 120 MBytes 8.88 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.11 sec 10.4 GBytes 8.79 Gbits/sec receiver




What am I doing wrong here?

 

[cvega]

Looks like _not_ setting the MTU does not set it to default value on Xigmanas. Now working fine..