Slow Network speed

[sajtos888]

Hello I am very new to using proxmox!
There would already be a network problem.
My structure is currently ISP (2000Mb / s)> physical PfSense> 10Gbit Switch> Proxmox
PfSense includes 1 2.5Gb / s network card (WAN) and one 10gbit network card (LAN) (x520-da1)
Proxmox includes i5-3740 16GB RAM 10Gbit Network Adapter (x520-da1)
The problem is that you don't have full speed on either speedtest-cli nor iperf3

PfSense iperf server
Proxmox iperf client

Proxmox does not run a guest machine

Supported ports: [ FIBRE ]
        Supported link modes:   10000baseT/Full
        Supported pause frame use: Symmetric
        Supports auto-negotiation: No
        Supported FEC modes: Not reported
        Advertised link modes:  10000baseT/Full
        Advertised pause frame use: Symmetric
        Advertised auto-negotiation: No
        Advertised FEC modes: Not reported
        Speed: 10000Mb/s
        Duplex: Full
        Auto-negotiation: off
        Port: FIBRE
        PHYAD: 0
        Transceiver: internal
        Supports Wake-on: d
        Wake-on: d
        Current message level: 0x00000007 (7)
                               drv probe link
        Link detected: yes

Settings for vmbr0:
        Supported ports: [  ]
        Supported link modes:   Not reported
        Supported pause frame use: No
        Supports auto-negotiation: No
        Supported FEC modes: Not reported
        Advertised link modes:  Not reported
        Advertised pause frame use: No
        Advertised auto-negotiation: No
        Advertised FEC modes: Not reported
        Speed: 10000Mb/s
        Duplex: Unknown! (255)
        Auto-negotiation: off
        Port: Other
        PHYAD: 0
        Transceiver: internal
        Link detected: yes

[  5] local 192.168.39.3 port 36748 connected to 192.168.39.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   380 MBytes  3.19 Gbits/sec    2    467 KBytes
[  5]   1.00-2.00   sec   379 MBytes  3.18 Gbits/sec   18    471 KBytes
[  5]   2.00-3.00   sec   377 MBytes  3.16 Gbits/sec   26    478 KBytes
[  5]   3.00-4.00   sec   376 MBytes  3.15 Gbits/sec   25    469 KBytes
[  5]   4.00-5.00   sec   378 MBytes  3.17 Gbits/sec   21    465 KBytes
[  5]   5.00-6.00   sec   378 MBytes  3.17 Gbits/sec   25    481 KBytes
[  5]   6.00-7.00   sec   377 MBytes  3.16 Gbits/sec   56    472 KBytes
[  5]   7.00-8.00   sec   379 MBytes  3.18 Gbits/sec   71    475 KBytes
[  5]   8.00-9.00   sec   378 MBytes  3.17 Gbits/sec   65    477 KBytes
[  5]   9.00-10.00  sec   377 MBytes  3.16 Gbits/sec   57    472 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  3.69 GBytes  3.17 Gbits/sec  366             sender
[  5]   0.00-10.00  sec  3.69 GBytes  3.17 Gbits/sec                  receiver

 

[Dunuin]

Network performance really depends on the power of your CPU. I guess your CPU is the bottleneck here and it just can't handle the amount of packets it needs to route. Using a default MTU of 1500 I also only get around 3-4Gbit. If I activate jumboframes in my switch and set all NICs to use a MTU of 9000 this will result in way less packets (only one big packet instead of 6 small ones) so the CPU gets less packets to route and performance increases to 10Gbit.

 

[sajtos888]

I temporarily changed the MTU to 9000 on both PfSense and Proxmox

[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   315 KBytes  2.58 Mbits/sec    3   8.74 KBytes
[  5]   1.00-2.00   sec  0.00 Bytes  0.00 bits/sec    1   8.74 KBytes
[  5]   2.00-3.00   sec  0.00 Bytes  0.00 bits/sec    0   8.74 KBytes
[  5]   3.00-4.00   sec  0.00 Bytes  0.00 bits/sec    1   8.74 KBytes
[  5]   4.00-5.00   sec  0.00 Bytes  0.00 bits/sec    0   8.74 KBytes
[  5]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec    0   8.74 KBytes
[  5]   6.00-7.00   sec  0.00 Bytes  0.00 bits/sec    1   8.74 KBytes
[  5]   7.00-8.00   sec  0.00 Bytes  0.00 bits/sec    0   8.74 KBytes
[  5]   8.00-9.00   sec  0.00 Bytes  0.00 bits/sec    0   8.74 KBytes
[  5]   9.00-10.00  sec  0.00 Bytes  0.00 bits/sec    0   8.74 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   315 KBytes   258 Kbits/sec    6             sender
[  5]   0.00-10.00  sec  0.00 Bytes  0.00 bits/sec                  receiver

 

[Dunuin]

Looks like the packets get lost somewhere. Really every part of the chain needs to work at 9000 MTU or your packets get stuck if something in between is still working with 1500 MTU.

Did you setup your pfsense, managed switch, bridges/nics of your PVE host to use a 9000 MTU?

 

[sajtos888]

So far, OpenMediaVault has been in the same configuration and the network has worked perfectly with it
OMV's download speed from the Internet was 2000Mbit / s provided by the service provider
Proxmoxon reaches about 900Mbit / s

Proxmox doesn't take advantage of internet speed either

The hardware didn’t just change the system and everything on OMV was good

 

[sajtos888]

Beállítottad a pfsense-t, a felügyelt kapcsolót, a PVE gazdagép hídjait/nic-jeit 9000 MTU használatára?

Yes!