[SOLVED] PBS benchmark tool - Good TLS speed in local but decreased TLS speed from remote

[TheVisitor92]

Hi there, as the title suggested, I would like to troubleshooting my issue with you. I installed a Proxmox Backup Server on a dedicated hardware:

HP Proliant DL380 Gen7: Intel Xeon E5620 2.40Ghz (4/8 C/T) AES supported and enabled, 16Gb RAM no-ECC, RAID5 2.50Tb Datastore (BackupPVE), NIC 1G ethernet

If I run proxmox-backup-client benchmark --repository BackupPVE i have this result:

Uploaded 422 chunks in 5 seconds.
Time per request: 12021 microseconds.
TLS speed: 348.91 MB/s
SHA256 speed: 203.24 MB/s
Compression speed: 277.73 MB/s
Decompress speed: 427.61 MB/s
AES256/GCM speed: 518.51 MB/s
Verify speed: 136.27 MB/s
┌───────────────────────────────────┬───────────────────┐
│ Name │ Value │
╞═══════════════════════════════════╪═══════════════════╡
│ TLS (maximal backup upload speed) │ 348.91 MB/s (28%) │
├───────────────────────────────────┼───────────────────┤
│ SHA256 checksum computation speed │ 203.24 MB/s (10%) │
├───────────────────────────────────┼───────────────────┤
│ ZStd level 1 compression speed │ 277.73 MB/s (37%) │
├───────────────────────────────────┼───────────────────┤
│ ZStd level 1 decompression speed │ 427.61 MB/s (36%) │
├───────────────────────────────────┼───────────────────┤
│ Chunk verification speed │ 136.27 MB/s (18%) │
├───────────────────────────────────┼───────────────────┤
│ AES256 GCM encryption speed │ 518.51 MB/s (14%) │
└───────────────────────────────────┴───────────────────┘

Now, from my main Proxmox server I get this result using proxmox-backup-client benchmark --repository root@pam@192.168.50.151:8007:BackupPVE:

Uploaded 148 chunks in 5 seconds.
Time per request: 35847 microseconds.
TLS speed: 117.00 MB/s
SHA256 speed: 441.57 MB/s
Compression speed: 505.09 MB/s
Decompress speed: 739.23 MB/s
AES256/GCM speed: 1642.18 MB/s
Verify speed: 270.36 MB/s
┌───────────────────────────────────┬────────────────────┐
│ Name │ Value │
╞═══════════════════════════════════╪════════════════════╡
│ TLS (maximal backup upload speed) │ 117.00 MB/s (9%) │
├───────────────────────────────────┼────────────────────┤
│ SHA256 checksum computation speed │ 441.57 MB/s (22%) │
├───────────────────────────────────┼────────────────────┤
│ ZStd level 1 compression speed │ 505.09 MB/s (67%) │
├───────────────────────────────────┼────────────────────┤
│ ZStd level 1 decompression speed │ 739.23 MB/s (62%) │
├───────────────────────────────────┼────────────────────┤
│ Chunk verification speed │ 270.36 MB/s (36%) │
├───────────────────────────────────┼────────────────────┤
│ AES256 GCM encryption speed │ 1642.18 MB/s (45%) │
└───────────────────────────────────┴────────────────────┘

The thing is that I ran several benchmarks from my PVE server, and the TLS speed never went above 117 MB/s the max was 117.83 MB/s.
I think that could be a sort of bottleneck, but I don't know where since it is a point to point setup with two 1G ethernet nics. I checked the PVE server's CPU usage too, and barely goes above 12% utilization with all the VMs up un running. The PBS's CPU has a good frequency too.

If can be helpful, this is the iperf3 test between PVE and PBS:

Connecting to host 192.168.50.151, port 5201
[ 5] local 192.168.50.156 port 52812 connected to 192.168.50.151 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 113 MBytes 950 Mbits/sec 38 212 KBytes
[ 5] 1.00-2.00 sec 113 MBytes 944 Mbits/sec 27 214 KBytes
[ 5] 2.00-3.00 sec 112 MBytes 939 Mbits/sec 41 206 KBytes
[ 5] 3.00-4.00 sec 112 MBytes 941 Mbits/sec 50 170 KBytes
[ 5] 4.00-5.00 sec 112 MBytes 941 Mbits/sec 41 164 KBytes
[ 5] 5.00-6.00 sec 113 MBytes 945 Mbits/sec 19 226 KBytes
[ 5] 6.00-7.00 sec 112 MBytes 941 Mbits/sec 33 211 KBytes
[ 5] 7.00-8.00 sec 112 MBytes 938 Mbits/sec 18 214 KBytes
[ 5] 8.00-9.00 sec 112 MBytes 942 Mbits/sec 16 212 KBytes
[ 5] 9.00-10.00 sec 112 MBytes 943 Mbits/sec 9 211 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 1.10 GBytes 942 Mbits/sec 292 sender
[ 5] 0.00-10.00 sec 1.10 GBytes 941 Mbits/sec receiver

Anyway, this could be the normal behavior of the setup, but I need clarifications, thanks!

 

[dcsapak]

you have 1 Gbit connection which has a theoretical maximum of 125MB/s so having 117 MB/s is not that bad (is about the 943 MBit/s you got with iperf; 943/8 = 117.875 )

 

[TheVisitor92]

you have 1 Gbit connection which has a theoretical maximum of 125MB/s so having 117 MB/s is not that bad (is about the 943 MBit/s you got with iperf; 943/8 = 117.875 )

Thank you, I'm feeling dumb, I was confusing MBit/s with MB/s... sorry. Thank you!