Remote sync very slow (0,4 MiB/s)

C

carsten2

Renowned Member
Mar 25, 2017
277
29
68
55
The remote sync is very slow. It transfers only 0.4 MiB/s and takes 1h20min to transfer 2GB. See the log.

Code: 
2020-11-15T09:50:18+01:00: sync snapshot "vm/101/2020-10-13T22:00:02Z"
2020-11-15T09:50:19+01:00: sync archive qemu-server.conf.blob
2020-11-15T09:50:19+01:00: sync archive drive-virtio0.img.fidx
2020-11-15T11:00:30+01:00: downloaded 1775574586 bytes (0.40 MiB/s)
2020-11-15T11:00:30+01:00: sync archive drive-virtio1.img.fidx
2020-11-15T11:12:32+01:00: downloaded 298909805 bytes (0.41 MiB/s)
2020-11-15T11:12:32+01:00: got backup log file "client.log.blob"
2020-11-15T11:12:32+01:00: sync snapshot "vm/101/2020-10-13T22:00:02Z" done

Also because the sync is that slow, it produces timeouts like the follwing
Code: 
2020-11-15T09:20:04+01:00: sync group vm/101 failed - authentication failed - invalid ticket - expired

BTW: Please change numbers to human readable numbers as suggested in a bugzilla ticket. 298909805 / 1775574586 is much less readable than 298 MiB / 1775 MiB.
 
Last edited:
carsten2 said:
The remote sync is very slow. It transfers only 0.4 MiB/s and takes 1h20min to transfer 2GB. See the log.
Click to expand...
Can you please describe the network between the two instances?
* Are you tunneling it through some layer? If yes, can you try without tunneling (or another more lightweight one) to see if that is the producing some of that effects.
* what are latencies like between those?
* what is the client benchmark output between those

carsten2 said:
Also because the sync is that slow, it produces timeouts like the follwing
Click to expand...
What versions are you on, as on 1.0 that should not happen (renewing the auth/ticket runs in an independent, periodic task there).
 
benchmark from remote site to local site:

local network
Uploaded 143 chunks in 5 seconds.
Time per request: 36770 microseconds.
TLS speed: 114.07 MB/s
SHA256 speed: 450.80 MB/s
Compression speed: 591.60 MB/s
Decompress speed: 1029.48 MB/s
AES256/GCM speed: 2179.01 MB/s
Verify speed: 317.33 MB/s

Remote to local over sshuttle (ssh-tunnel)
Uploaded 10 chunks in 182 seconds.
Time per request: 18227215 microseconds.
TLS speed: 0.23 MB/s
SHA256 speed: 460.78 MB/s
Compression speed: 602.34 MB/s
Decompress speed: 1039.39 MB/s
AES256/GCM speed: 2176.49 MB/s
Verify speed: 318.10 MB/s

Remote to local directly over port 8007
Uploaded 10 chunks in 119 seconds.
Time per request: 11966427 microseconds.
TLS speed: 0.35 MB/s
SHA256 speed: 461.34 MB/s
Compression speed: 598.53 MB/s
Decompress speed: 1032.67 MB/s
AES256/GCM speed: 2180.20 MB/s
Verify speed: 319.37 MB/s

How to test TCP-latency?

I am running 1.0 (but don't know exactly, when the update took place). I will see whether the error appears again.
 
Last edited:
Sshuttle (while being awesome) is notoriously slow. You might consider using a „real“ VPN like wireguard, those should achieve more throughout.
 
Cookiefamily said:
Sshuttle (while being awesome) is notoriously slow. You might consider using a „real“ VPN like wireguard, those should achieve more throughout.
Click to expand...
It is not sshuttle because the direct connection via port 8007 (even no VPN) is as slow as the sshuttle connection.
 
carsten2 said:
How to test TCP-latency?
Click to expand...
ping would help as a basic starter.

carsten2 said:
It is not sshuttle because the direct connection via port 8007 (even no VPN) is as slow as the sshuttle connection.
Click to expand...

I mean the non-tunneled connection needs about a minute less time to complete, so there's that...
 
Ping is 50ms (offsite over internet)
Yes, direct connection is faster, but still incredibly slow.

BTW: The reasons I use SSH is:
a) I dont't want to expose the GUI and the fact, that there is a PBS running. Also a complete UI-interface offers more attack surface.
b) With SSH I get fail2ban and other firewall attach filtering, which seems to be missing with PBS direct connection. At least fail2ban should be supported out of the box by PBS.
 
Last edited:
carsten2 said:
Ping is 50ms (offsite over internet)
Yes, direct connection is faster, but still incredibly slow.
Click to expand...
What is your upload bandwidth on the source, and the download bandwidth on the target?
Or, what does an iperf test says about the maximal practical bandwidth available?

carsten2 said:
a) I dont't want to expose the GUI and the fact, that there is a PBS running. Also a complete UI-interface offers more attack surface.
Click to expand...
No, not really, it's always just using the API and so it's reduced to the attack surface of the API.
It does not help for security at all, but it's your setup after all, just noting :)

carsten2 said:
b) With SSH I get fail2ban and other firewall attach filtering, which seems to be missing with PBS direct connection. At least fail2ban should be supported out of the box by PBS.
Click to expand...
You can already setup fail2ban on /var/log/proxmox-backup/api/auth.log, it is formatted such that the PVE docs should only need a few lightweight changes: https://pve.proxmox.com/wiki/Fail2ban
 
t.lamprecht said:
You can already setup fail2ban on /var/log/proxmox-backup/api/auth.log, it is formatted such that the PVE docs should only need a few lightweight changes: https://pve.proxmox.com/wiki/Fail2ban
Click to expand...
PBS should come with preconfigured fail2ban support. No service should be exposed to the internet without it. This is one reason I choose ssh tunnel, because everything with security works out of the box.
 
Has anything been figured out on this issue?

I'm dealing with a similar situation. I'm directly connected with no tunneling and getting around .80MiB/s. The slowest link in this connection is the download speed at the remote site, which is a theoretical 5MiB/s

Thanks,
Daniel
 
dbayer said:
Bump.
Click to expand...
please do not bump threads without any additional information

dbayer said:
I'm dealing with a similar situation. I'm directly connected with no tunneling and getting around .80MiB/s. The slowest link in this connection is the download speed at the remote site, which is a theoretical 5MiB/s
Click to expand...
what is the latency in between? can you test the bandwidth with iperf?
 
i have also a very slow sync speed running Backup Server 1.0-6 on both ends.
network connection 1000mb down 50mb up

no matter if i use the connection over wireguard (as shown below) or over the direct network. The speed is very slow regardless of the connection.
very confusing for me was, that it sometimes download at full speed with about 40mb/s
and then all over sudden it dropped to maybe 1mb/s

when i test wget and download a file from the same destination i reach 40mb/s +

hope that helps to investigate this

root@pbs:~# ping 192.168.159.228 PING 192.168.159.228 (192.168.159.228) 56(84) bytes of data. 64 bytes from 192.168.159.228: icmp_seq=1 ttl=62 time=27.5 ms 64 bytes from 192.168.159.228: icmp_seq=2 ttl=62 time=41.3 ms 64 bytes from 192.168.159.228: icmp_seq=3 ttl=62 time=33.0 ms 64 bytes from 192.168.159.228: icmp_seq=4 ttl=62 time=31.6 ms 64 bytes from 192.168.159.228: icmp_seq=5 ttl=62 time=28.8 ms 64 bytes from 192.168.159.228: icmp_seq=6 ttl=62 time=25.8 ms

Code: 
root@pbs:~# proxmox-backup-client benchmark --repository 192.168.159.228:Backup
Uploaded 13 chunks in 16 seconds.
Time per request: 1300976 microseconds.
TLS speed: 3.22 MB/s
SHA256 speed: 427.05 MB/s
Compression speed: 653.46 MB/s
Decompress speed: 1113.60 MB/s
AES256/GCM speed: 2441.89 MB/s
Verify speed: 300.89 MB/s
┌───────────────────────────────────┬────────────────────┐
│ Name                              │ Value              │
╞═══════════════════════════════════╪════════════════════╡
│ TLS (maximal backup upload speed) │ 3.22 MB/s (0%)     │
├───────────────────────────────────┼────────────────────┤
│ SHA256 checksum computation speed │ 427.05 MB/s (21%)  │
├───────────────────────────────────┼────────────────────┤
│ ZStd level 1 compression speed    │ 653.46 MB/s (87%)  │
├───────────────────────────────────┼────────────────────┤
│ ZStd level 1 decompression speed  │ 1113.60 MB/s (93%) │
├───────────────────────────────────┼────────────────────┤
│ Chunk verification speed          │ 300.89 MB/s (40%)  │
├───────────────────────────────────┼────────────────────┤
│ AES256 GCM encryption speed       │ 2441.89 MB/s (67%) │
└───────────────────────────────────┴────────────────────┘

Code: 
ProxmoxBackup Server 1.0-6
CPU
Memory
Root Disk
No valid subscription
You do not have a valid subscription for this server. Please visit www.proxmox.com to get a list of available options.
()
2021-02-08T16:46:56+01:00: starting new backup reader datastore 'Backup': "/mnt/Backup/"
2021-02-08T16:46:56+01:00: protocol upgrade done
2021-02-08T16:46:56+01:00: GET /download
2021-02-08T16:46:56+01:00: download "/mnt/Backup/vm/242/2021-02-07T18:18:25Z/index.json.blob"
2021-02-08T16:46:56+01:00: GET /download
2021-02-08T16:46:56+01:00: download "/mnt/Backup/vm/242/2021-02-07T18:18:25Z/drive-virtio1.img.fidx"
2021-02-08T16:46:56+01:00: register chunks in 'drive-virtio1.img.fidx' as downloadable.
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/4d15/4d15dd406b5cc4f803aef79759089e787e3353a5283f69adce6296e4e7bfaf22"
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/c2c6/c2c60f3858faf207f9dfd07ad63f18b788f0a51f356330c43c01b9d1cb0f8ec8"
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/9815/981576fac0d4c7b96e44360a78fcbb12e152ae73cdbce3c2d60e1470812ba6fb"
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/9a8f/9a8f63a8f7fbb97b80b36da1fdf37bcb4814a799b6a4dc75a0e42fd982e0213b"
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/7eaa/7eaa1a3c3193cd9e85ae91870e45541949610b0ab1703383f0e2f354a230d7f6"
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/f7ab/f7abdafa6aed0bd7e21c331bc5fc5f36f72620eb148387a161294dd998fc005e"
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/1983/19836318f89c2ca299bda016d36b268052fe5540387511ebcd0431d20254b0f5"
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/47bc/47bcb02c0da3df09b393b54c2eaf5015af32c91386ff80ab7a8c469054798b9e"
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/5ea3/5ea32a161291e1b8e7eaf1be454d27bda7851844541d5474d639f5507ded97a2"
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/ae05/ae05a951edbd1efc62592eb6cc6b968498c47ba2b050a7801e5ebb174b039c1c"
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/edc0/edc042a6c13067d3f29c6961187029bb37562f1da696816560a00fef900a8aa3"
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/49ef/49ef10c715bd0d1e009f57025aff5b36fe5debea3f99825a4149e7a9b2bb1fef"
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/91d8/91d8126896f38244ed36ee161fa898e7243be83f5832e2349dbac146c0dd04e7"
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/9782/978244081548e75f7c327fd03e3cc245da812b34062fcf2e16e7042ee93625f8"
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/2b24/2b24bbbfcca9508b9fa8bfd26da2adee6e2544ec479eb0100b427e9c856fc62f"
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/6c71/6c71fcb77ddd9e4e47cfd6b986d6d6ed37ab80b16e5c7e9c8a564779ede63694"
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/3b1e/3b1e2e227b78cd858757cf2413ba1e1fb56de10307df9c2f028bca9e9780e914"
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/0df3/0df3c36aaadd2c1dff919667dd2513ba6350814b24a611898ae27ae43a007090"
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/3b2f/3b2fbf6963834115c1555fea450c0ee58f26e40b508cf1c6bc69897a135a67bd"
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/d845/d84585c515a3ee29894c615254629b89569c2ae2457aa13c56296499971c6817"
2021-02-08T16:47:11+01:00: GET /chunk
2021-02-08T16:47:11+01:00: download chunk "/mnt/Backup/.chunks/5479/547904dccb456bb5eedba8c6692af7337c815c49c25fc7ea6d0dfbd6a36d8c51"
2021-02-08T16:47:12+01:00: GET /chunk
2021-02-08T16:47:12+01:00: download chunk "/mnt/Backup/.chunks/e7c6/e7c60accd2837c6d5e8645e5fdb3f41fe1fad3a2af74c79967b949bd05a60a26"
2021-02-08T16:47:13+01:00: GET /chunk
2021-02-08T16:47:13+01:00: download chunk "/mnt/Backup/.chunks/5bc5/5bc5fe8c28745145908f2e5af277377a2bfd970881698d27921362dc088b3511"
2021-02-08T16:47:14+01:00: GET /chunk
2021-02-08T16:47:14+01:00: download chunk "/mnt/Backup/.chunks/939c/939c8463f1fadfeb9f6dc6269ecbaac07bee61ffbc65ce0d490a4af9ccac4801"
2021-02-08T16:47:15+01:00: GET /chunk
2021-02-08T16:47:15+01:00: download chunk "/mnt/Backup/.chunks/3a1b/3a1b60198eea7fe645737d3f2d1c3fe2aef508ea5aea736aa9b7ab494620bd44"
2021-02-08T16:47:15+01:00: GET /chunk
2021-02-08T16:47:15+01:00: download chunk "/mnt/Backup/.chunks/5d47/5d479910b31d73f53956a863f5efc86132a4a18a6b82bdc6e3654fc09ea5fc8c"
2021-02-08T16:47:16+01:00: GET /chunk
2021-02-08T16:47:16+01:00: download chunk "/mnt/Backup/.chunks/d54f/d54f476378f7eed1914885d60f88c3e468d1ddb5465afb001edab583677f17fa"
2021-02-08T16:47:16+01:00: GET /chunk
2021-02-08T16:47:16+01:00: download chunk "/mnt/Backup/.chunks/862d/862de2a338cc2fbdeee376b123a8f7e47548ec392e7b642403af347898562153"
2021-02-08T16:47:17+01:00: GET /chunk
2021-02-08T16:47:17+01:00: download chunk "/mnt/Backup/.chunks/684a/684ab9b68ac3c0d1d9eae344ae7614fceca754297a29bc5318536ab8fa602d9a"
2021-02-08T16:47:17+01:00: GET /chunk
2021-02-08T16:47:17+01:00: download chunk "/mnt/Backup/.chunks/2a48/2a4812921946167b1d1fa4200b71d45701fa92292c1c168e213ea6e74bad5581"
2021-02-08T16:47:18+01:00: GET /chunk
2021-02-08T16:47:18+01:00: download chunk "/mnt/Backup/.chunks/d9b7/d9b7d75b74f48f4ac77f5598f1ebeb8aa855752eabba1c692ad6f274b388856f"
2021-02-08T16:47:18+01:00: GET /chunk
2021-02-08T16:47:18+01:00: download chunk "/mnt/Backup/.chunks/9e40/9e40d589fb98f02d186f2c7dc4d52bb0bf3b29514dbf06c8110d85d0dd31f7f1"
2021-02-08T16:47:19+01:00: GET /chunk
2021-02-08T16:47:19+01:00: download chunk "/mnt/Backup/.chunks/d4b4/d4b4a5df1e96de176e9fbba507827d4f4885395002017bc3bec1b3d5fe1977e6"
2021-02-08T16:47:19+01:00: GET /chunk
2021-02-08T16:47:19+01:00: download chunk "/mnt/Backup/.chunks/9aa5/9aa57e79f966095acb429788eecd54f027199060db70dd0ff6e6874f49486740"
2021-02-08T16:47:19+01:00: GET /chunk
2021-02-08T16:47:19+01:00: download chunk "/mnt/Backup/.chunks/8c31/8c316f9b2f3471985c17324f815bd4653270b1666af256aadc7c27cbaae03207"
2021-02-08T16:47:20+01:00: GET /chunk
2021-02-08T16:47:20+01:00: download chunk "/mnt/Backup/.chunks/1830/1830f83a44bfa3cfacfbfe0daac79d27e91063e6b738c8984ee5560e16f9b84b"
2021-02-08T16:47:20+01:00: GET /chunk
2021-02-08T16:47:20+01:00: download chunk "/mnt/Backup/.chunks/928a/928a6b8068ca3668d80692f56720e150d7cb0178923c51b5606cc7c604343aa8"
2021-02-08T16:47:21+01:00: GET /chunk
2021-02-08T16:47:21+01:00: download chunk "/mnt/Backup/.chunks/2afd/2afdbca1ec266e32e8ec36fe1da712e84d5a78e74eafff53ac107c14f430b021"
2021-02-08T16:47:21+01:00: GET /chunk
2021-02-08T16:47:21+01:00: download chunk "/mnt/Backup/.chunks/4d5a/4d5aa6a19ae72416f479729f01eb913f44986265b117b14dd3e5cf3068e2a050"
2021-02-08T16:47:21+01:00: GET /chunk
2021-02-08T16:47:21+01:00: download chunk "/mnt/Backup/.chunks/8f04/8f047b0f191ae2d6dc4a6838f3b4504dafdfb165a7af31e527b8f48ff4d8e9e9"
2021-02-08T16:47:22+01:00: GET /chunk
2021-02-08T16:47:22+01:00: download chunk "/mnt/Backup/.chunks/47f5/47f5060c2655809f38f7e4b3fb3c5064071d7a3aa6e335efbf5ea32364a554f0"
2021-02-08T16:47:22+01:00: GET /chunk
2021-02-08T16:47:22+01:00: download chunk "/mnt/Backup/.chunks/883e/883ec8843ab6c1fd62ef1a20f6485f362ac9d9faa61476f1db2371c4c6b76481"
2021-02-08T16:47:23+01:00: GET /chunk
2021-02-08T16:47:23+01:00: download chunk "/mnt/Backup/.chunks/4dcd/4dcdf5559be44289940261ff5152eae315a0ca8b8577f1ecfaa29f22fecdbee8"
2021-02-08T16:47:23+01:00: GET /chunk
2021-02-08T16:47:23+01:00: download chunk "/mnt/Backup/.chunks/47cb/47cb1a729ac5e0643e0be4d3b7fa3a5f72ef98c05a081ae75433dc38db37005f"
2021-02-08T16:47:24+01:00: GET /chunk
2021-02-08T16:47:24+01:00: download chunk "/mnt/Backup/.chunks/5491/54914143ddf5ded8ad23cb1d3692209fc5c79edb999015f6d5831f3e12833094"
2021-02-08T16:47:24+01:00: GET /chunk
2021-02-08T16:47:24+01:00: download chunk "/mnt/Backup/.chunks/18ba/18baf6f8a59e70f5ea8c98b24c761d263a491032202374f8bfd9be919436a975"
2021-02-08T16:47:25+01:00: GET /chunk
2021-02-08T16:47:25+01:00: download chunk "/mnt/Backup/.chunks/de92/de92d11590cafb3532b162acf5723c087bf851c6cd9a575c1a0a69ee3166530d"
2021-02-08T16:47:25+01:00: GET /chunk
2021-02-08T16:47:25+01:00: download chunk "/mnt/Backup/.chunks/8c8d/8c8d25bd3dbd291e3d3ba40c01ef5ac03311148e59f99526065f4c45e293bdb5"
2021-02-08T16:47:25+01:00: GET /chunk
2021-02-08T16:47:25+01:00: download chunk "/mnt/Backup/.chunks/45c8/45c8ac41a569151a6a8c87b0be9b7baecb9143a5e7b0e4e96eb59096bdeab1d6"
2021-02-08T16:47:26+01:00: GET /chunk
2021-02-08T16:47:26+01:00: download chunk "/mnt/Backup/.chunks/c295/c295ac13dfb4aee468673bbbe40ba0a7a5d02c313fb9c35e07e5b031491cbb5d"
2021-02-08T16:47:26+01:00: GET /chunk
2021-02-08T16:47:26+01:00: download chunk "/mnt/Backup/.chunks/f530/f5308201fe7492d379d06bd817b7d40f2b097ea154b0d3cdeec3cf71d1e6b563"
2021-02-08T16:47:27+01:00: GET /chunk
2021-02-08T16:47:27+01:00: download chunk "/mnt/Backup/.chunks/5dc7/5dc7382ade94fdd58f70e24cde1b463a6eedd490f35aa702b7ee150693364b17"
2021-02-08T16:47:27+01:00: GET /chunk
2021-02-08T16:47:27+01:00: download chunk "/mnt/Backup/.chunks/8b3c/8b3cb36b5bc9a64142325525d94fc18020582f394cb871d94055a7d67bd9edef"
2021-02-08T16:47:28+01:00: GET /chunk
2021-02-08T16:47:28+01:00: download chunk "/mnt/Backup/.chunks/0200/02007b3c5224fe28cb3fd97234f6ddb0a4a091d50fceca152a9b3317afcc5a40"
2021-02-08T16:47:28+01:00: GET /chunk
2021-02-08T16:47:28+01:00: download chunk "/mnt/Backup/.chunks/64e5/64e544f495f43eee0075bef602f50adf091bacb63e7eed37bd94fac22c0c936f"
2021-02-08T16:47:29+01:00: GET /chunk
2021-02-08T16:47:29+01:00: download chunk "/mnt/Backup/.chunks/fd02/fd02b7c583563c69e27da625a1cfbf8613d8136a0100ce128cb299f60b7596c6"
2021-02-08T16:47:29+01:00: GET /chunk
2021-02-08T16:47:29+01:00: download chunk "/mnt/Backup/.chunks/f6c7/f6c7a86a2d050539098c2de1532156ed3e88c9776484699458af7fbe0f31cb25"
2021-02-08T16:47:29+01:00: GET /chunk
2021-02-08T16:47:29+01:00: download chunk "/mnt/Backup/.chunks/5cc7/5cc745db8e4c4fd799ff6b247cccaab5524900099a57611fc3dbf505f45d90c2"
2021-02-08T16:47:30+01:00: GET /chunk
2021-02-08T16:47:30+01:00: download chunk "/mnt/Backup/.chunks/c2f7/c2f7bbe98531a504194e08f1f684ed67dadb770067d4c4d6d89b05435ae65372"
2021-02-08T16:47:30+01:00: GET /chunk
2021-02-08T16:47:30+01:00: download chunk "/mnt/Backup/.chunks/674d/674d90371a8593f9dd1bd73a682ea399c6fdf0780a3f53755646863dd0c1676c"
2021-02-08T16:47:31+01:00: GET /chunk
2021-02-08T16:47:31+01:00: download chunk "/mnt/Backup/.chunks/874c/874c1be5541cc8b670efbd850da3769d6b72aaa28e3fd4461ca703de62504bff"
2021-02-08T16:47:31+01:00: GET /chunk
2021-02-08T16:47:31+01:00: download chunk "/mnt/Backup/.chunks/af75/af7557b43e1c104460a9ff52d536e4b27a19738e9cebb477e14f1004557ce33d"
2021-02-08T16:47:32+01:00: GET /chunk
2021-02-08T16:47:32+01:00: download chunk "/mnt/Backup/.chunks/c7eb/c7eb0ddcb8f30dc4641b4780bd537f4c37a9d9231bb372195ffd5c002110bac1"
2021-02-08T16:47:32+01:00: GET /chunk
2021-02-08T16:47:32+01:00: download chunk "/mnt/Backup/.chunks/3aab/3aabbb4e6b602f465074d6f23eec215bcca2d116e9d288226b530b8cbe60b46e"
 
Last edited:
tommytom said:
network connection 1000mb down 50mb up
Click to expand...
are you talking mbyte or mbit ? for mbit, the result does not look too bad since 3.22MiB -> 25MBit/s so roughly half

please post an iperf test in both directions and also 'proxmox-backup-client benchmark' in both directions
 
dcsapak said:
are you talking mbyte or mbit ? for mbit, the result does not look too bad since 3.22MiB -> 25MBit/s so roughly half

please post an iperf test in both directions and also 'proxmox-backup-client benchmark' in both directions
Click to expand...
to clarify: i have 1 gigabit downstream and 50mbit Upstream on my home sync PBS site which produces slow speed.

the main backup-server server is connected with 10gigabit to the network 30-50ms ping time away

as i wrote.. sometimes i had even 40megabyte download speed. before it dropped on a very huge vm with 1.2Tb disk size.
 
Last edited:
tommytom said:
to clarify: i have 1 gigabit downstream and 50mbit Upstream on my home sync PBS site which produces slow speed.

the main backup-server server is connected with 10gigabit to the network 30-50ms ping time away
Click to expand...
it is still not clear in which direction you did benchmark, also does the backup-server really have a 10gbit uplink?

tommytom said:
as i wrote.. sometimes i had even 40megabyte download speed. before it dropped on a very huge vm with 1.2Tb disk size.
Click to expand...
this also very much depends on the storage + caches, e.g. if you have 'slow' mechanical disks and the chunks are not in the memory cache, speeds will be much lower
this: https://git.proxmox.com/?p=proxmox-...0f58d259606b643df542e2664b08009fe;hb=HEAD#l64
has some napkin math calculations for how much speed you can get, but of course higher latency links will be detrimental to speed
 
False alarm!
It's because of our network connection. We have a 1GigaBit line with Vodafone as a business customer. Unbelievably, they throttle the line down to 10mbit during the day. Shortly after midnight, everything works at full speed. Then you also get a good performance with the PBS as expected.

I cannot recommend Vodafone in Germany. We will now probably get a line with Colt.

Thanks for your great work !
 
  • Like
Reactions: larsen, Dunuin, DerDanilo and 2 others
tommytom said:
False alarm!
It's because of our network connection. We have a 1GigaBit line with Vodafone as a business customer. Unbelievably, they throttle the line down to 10mbit during the day. Shortly after midnight, everything works at full speed. Then you also get a good performance with the PBS as expected.

I cannot recommend Vodafone in Germany. We will now probably get a line with Colt.

Thanks for your great work !
Click to expand...
How did you discover they were throttling?
 
  • Like
Reactions: DerDanilo
because i checked the line frequently and found out, that single line connections like with wget a throttled in the evening hours and get back to fullspeed short after 00:00 am - every day... from up to 50mbyte/s they throttle it to 500kb-800kb/s
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back