Remote sync very slow (0,4 MiB/s)

carsten2

Active Member
Mar 25, 2017
223
14
38
53
The remote sync is very slow. It transfers only 0.4 MiB/s and takes 1h20min to transfer 2GB. See the log.

Code:
2020-11-15T09:50:18+01:00: sync snapshot "vm/101/2020-10-13T22:00:02Z"
2020-11-15T09:50:19+01:00: sync archive qemu-server.conf.blob
2020-11-15T09:50:19+01:00: sync archive drive-virtio0.img.fidx
2020-11-15T11:00:30+01:00: downloaded 1775574586 bytes (0.40 MiB/s)
2020-11-15T11:00:30+01:00: sync archive drive-virtio1.img.fidx
2020-11-15T11:12:32+01:00: downloaded 298909805 bytes (0.41 MiB/s)
2020-11-15T11:12:32+01:00: got backup log file "client.log.blob"
2020-11-15T11:12:32+01:00: sync snapshot "vm/101/2020-10-13T22:00:02Z" done

Also because the sync is that slow, it produces timeouts like the follwing
Code:
2020-11-15T09:20:04+01:00: sync group vm/101 failed - authentication failed - invalid ticket - expired

BTW: Please change numbers to human readable numbers as suggested in a bugzilla ticket. 298909805 / 1775574586 is much less readable than 298 MiB / 1775 MiB.
 
Last edited:

t.lamprecht

Proxmox Staff Member
Staff member
Jul 28, 2015
5,504
1,753
164
South Tyrol/Italy
shop.proxmox.com
The remote sync is very slow. It transfers only 0.4 MiB/s and takes 1h20min to transfer 2GB. See the log.
Can you please describe the network between the two instances?
* Are you tunneling it through some layer? If yes, can you try without tunneling (or another more lightweight one) to see if that is the producing some of that effects.
* what are latencies like between those?
* what is the client benchmark output between those

Also because the sync is that slow, it produces timeouts like the follwing
What versions are you on, as on 1.0 that should not happen (renewing the auth/ticket runs in an independent, periodic task there).
 

carsten2

Active Member
Mar 25, 2017
223
14
38
53
benchmark from remote site to local site:

local network
Uploaded 143 chunks in 5 seconds.
Time per request: 36770 microseconds.
TLS speed: 114.07 MB/s
SHA256 speed: 450.80 MB/s
Compression speed: 591.60 MB/s
Decompress speed: 1029.48 MB/s
AES256/GCM speed: 2179.01 MB/s
Verify speed: 317.33 MB/s

Remote to local over sshuttle (ssh-tunnel)
Uploaded 10 chunks in 182 seconds.
Time per request: 18227215 microseconds.
TLS speed: 0.23 MB/s
SHA256 speed: 460.78 MB/s
Compression speed: 602.34 MB/s
Decompress speed: 1039.39 MB/s
AES256/GCM speed: 2176.49 MB/s
Verify speed: 318.10 MB/s

Remote to local directly over port 8007
Uploaded 10 chunks in 119 seconds.
Time per request: 11966427 microseconds.
TLS speed: 0.35 MB/s
SHA256 speed: 461.34 MB/s
Compression speed: 598.53 MB/s
Decompress speed: 1032.67 MB/s
AES256/GCM speed: 2180.20 MB/s
Verify speed: 319.37 MB/s

How to test TCP-latency?

I am running 1.0 (but don't know exactly, when the update took place). I will see whether the error appears again.
 
Last edited:

Cookiefamily

Active Member
Jan 29, 2020
132
36
33
Germany
Sshuttle (while being awesome) is notoriously slow. You might consider using a „real“ VPN like wireguard, those should achieve more throughout.
 

carsten2

Active Member
Mar 25, 2017
223
14
38
53
Sshuttle (while being awesome) is notoriously slow. You might consider using a „real“ VPN like wireguard, those should achieve more throughout.
It is not sshuttle because the direct connection via port 8007 (even no VPN) is as slow as the sshuttle connection.
 

t.lamprecht

Proxmox Staff Member
Staff member
Jul 28, 2015
5,504
1,753
164
South Tyrol/Italy
shop.proxmox.com
How to test TCP-latency?
ping would help as a basic starter.

It is not sshuttle because the direct connection via port 8007 (even no VPN) is as slow as the sshuttle connection.

I mean the non-tunneled connection needs about a minute less time to complete, so there's that...
 

carsten2

Active Member
Mar 25, 2017
223
14
38
53
Ping is 50ms (offsite over internet)
Yes, direct connection is faster, but still incredibly slow.

BTW: The reasons I use SSH is:
a) I dont't want to expose the GUI and the fact, that there is a PBS running. Also a complete UI-interface offers more attack surface.
b) With SSH I get fail2ban and other firewall attach filtering, which seems to be missing with PBS direct connection. At least fail2ban should be supported out of the box by PBS.
 
Last edited:

t.lamprecht

Proxmox Staff Member
Staff member
Jul 28, 2015
5,504
1,753
164
South Tyrol/Italy
shop.proxmox.com
Ping is 50ms (offsite over internet)
Yes, direct connection is faster, but still incredibly slow.
What is your upload bandwidth on the source, and the download bandwidth on the target?
Or, what does an iperf test says about the maximal practical bandwidth available?

a) I dont't want to expose the GUI and the fact, that there is a PBS running. Also a complete UI-interface offers more attack surface.
No, not really, it's always just using the API and so it's reduced to the attack surface of the API.
It does not help for security at all, but it's your setup after all, just noting :)

b) With SSH I get fail2ban and other firewall attach filtering, which seems to be missing with PBS direct connection. At least fail2ban should be supported out of the box by PBS.
You can already setup fail2ban on /var/log/proxmox-backup/api/auth.log, it is formatted such that the PVE docs should only need a few lightweight changes: https://pve.proxmox.com/wiki/Fail2ban
 

carsten2

Active Member
Mar 25, 2017
223
14
38
53
You can already setup fail2ban on /var/log/proxmox-backup/api/auth.log, it is formatted such that the PVE docs should only need a few lightweight changes: https://pve.proxmox.com/wiki/Fail2ban
PBS should come with preconfigured fail2ban support. No service should be exposed to the internet without it. This is one reason I choose ssh tunnel, because everything with security works out of the box.
 

dbayer

Active Member
Apr 15, 2016
63
8
28
52
Has anything been figured out on this issue?

I'm dealing with a similar situation. I'm directly connected with no tunneling and getting around .80MiB/s. The slowest link in this connection is the download speed at the remote site, which is a theoretical 5MiB/s

Thanks,
Daniel
 

dcsapak

Proxmox Staff Member
Staff member
Feb 1, 2016
8,513
1,097
164
34
Vienna
please do not bump threads without any additional information

I'm dealing with a similar situation. I'm directly connected with no tunneling and getting around .80MiB/s. The slowest link in this connection is the download speed at the remote site, which is a theoretical 5MiB/s
what is the latency in between? can you test the bandwidth with iperf?
 

tommytom

Active Member
Aug 22, 2015
32
5
28
i have also a very slow sync speed running Backup Server 1.0-6 on both ends.
network connection 1000mb down 50mb up

no matter if i use the connection over wireguard (as shown below) or over the direct network. The speed is very slow regardless of the connection.
very confusing for me was, that it sometimes download at full speed with about 40mb/s
and then all over sudden it dropped to maybe 1mb/s

when i test wget and download a file from the same destination i reach 40mb/s +

hope that helps to investigate this

root@pbs:~# ping 192.168.159.228 PING 192.168.159.228 (192.168.159.228) 56(84) bytes of data. 64 bytes from 192.168.159.228: icmp_seq=1 ttl=62 time=27.5 ms 64 bytes from 192.168.159.228: icmp_seq=2 ttl=62 time=41.3 ms 64 bytes from 192.168.159.228: icmp_seq=3 ttl=62 time=33.0 ms 64 bytes from 192.168.159.228: icmp_seq=4 ttl=62 time=31.6 ms 64 bytes from 192.168.159.228: icmp_seq=5 ttl=62 time=28.8 ms 64 bytes from 192.168.159.228: icmp_seq=6 ttl=62 time=25.8 ms

Code:
root@pbs:~# proxmox-backup-client benchmark --repository 192.168.159.228:Backup
Uploaded 13 chunks in 16 seconds.
Time per request: 1300976 microseconds.
TLS speed: 3.22 MB/s
SHA256 speed: 427.05 MB/s
Compression speed: 653.46 MB/s
Decompress speed: 1113.60 MB/s
AES256/GCM speed: 2441.89 MB/s
Verify speed: 300.89 MB/s
┌───────────────────────────────────┬────────────────────┐
│ Name                              │ Value              │
╞═══════════════════════════════════╪════════════════════╡
│ TLS (maximal backup upload speed) │ 3.22 MB/s (0%)     │
├───────────────────────────────────┼────────────────────┤
│ SHA256 checksum computation speed │ 427.05 MB/s (21%)  │
├───────────────────────────────────┼────────────────────┤
│ ZStd level 1 compression speed    │ 653.46 MB/s (87%)  │
├───────────────────────────────────┼────────────────────┤
│ ZStd level 1 decompression speed  │ 1113.60 MB/s (93%) │
├───────────────────────────────────┼────────────────────┤
│ Chunk verification speed          │ 300.89 MB/s (40%)  │
├───────────────────────────────────┼────────────────────┤
│ AES256 GCM encryption speed       │ 2441.89 MB/s (67%) │
└───────────────────────────────────┴────────────────────┘

Code:
ProxmoxBackup Server 1.0-6
CPU
Memory
Root Disk
No valid subscription
You do not have a valid subscription for this server. Please visit www.proxmox.com to get a list of available options.
()
2021-02-08T16:46:56+01:00: starting new backup reader datastore 'Backup': "/mnt/Backup/"
2021-02-08T16:46:56+01:00: protocol upgrade done
2021-02-08T16:46:56+01:00: GET /download
2021-02-08T16:46:56+01:00: download "/mnt/Backup/vm/242/2021-02-07T18:18:25Z/index.json.blob"
2021-02-08T16:46:56+01:00: GET /download
2021-02-08T16:46:56+01:00: download "/mnt/Backup/vm/242/2021-02-07T18:18:25Z/drive-virtio1.img.fidx"
2021-02-08T16:46:56+01:00: register chunks in 'drive-virtio1.img.fidx' as downloadable.
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/4d15/4d15dd406b5cc4f803aef79759089e787e3353a5283f69adce6296e4e7bfaf22"
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/c2c6/c2c60f3858faf207f9dfd07ad63f18b788f0a51f356330c43c01b9d1cb0f8ec8"
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/9815/981576fac0d4c7b96e44360a78fcbb12e152ae73cdbce3c2d60e1470812ba6fb"
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/9a8f/9a8f63a8f7fbb97b80b36da1fdf37bcb4814a799b6a4dc75a0e42fd982e0213b"
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/7eaa/7eaa1a3c3193cd9e85ae91870e45541949610b0ab1703383f0e2f354a230d7f6"
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/f7ab/f7abdafa6aed0bd7e21c331bc5fc5f36f72620eb148387a161294dd998fc005e"
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/1983/19836318f89c2ca299bda016d36b268052fe5540387511ebcd0431d20254b0f5"
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/47bc/47bcb02c0da3df09b393b54c2eaf5015af32c91386ff80ab7a8c469054798b9e"
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/5ea3/5ea32a161291e1b8e7eaf1be454d27bda7851844541d5474d639f5507ded97a2"
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/ae05/ae05a951edbd1efc62592eb6cc6b968498c47ba2b050a7801e5ebb174b039c1c"
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/edc0/edc042a6c13067d3f29c6961187029bb37562f1da696816560a00fef900a8aa3"
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/49ef/49ef10c715bd0d1e009f57025aff5b36fe5debea3f99825a4149e7a9b2bb1fef"
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/91d8/91d8126896f38244ed36ee161fa898e7243be83f5832e2349dbac146c0dd04e7"
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/9782/978244081548e75f7c327fd03e3cc245da812b34062fcf2e16e7042ee93625f8"
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/2b24/2b24bbbfcca9508b9fa8bfd26da2adee6e2544ec479eb0100b427e9c856fc62f"
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/6c71/6c71fcb77ddd9e4e47cfd6b986d6d6ed37ab80b16e5c7e9c8a564779ede63694"
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/3b1e/3b1e2e227b78cd858757cf2413ba1e1fb56de10307df9c2f028bca9e9780e914"
2021-02-08T16:47:10+01:00: GET /chunk
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/0df3/0df3c36aaadd2c1dff919667dd2513ba6350814b24a611898ae27ae43a007090"
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/3b2f/3b2fbf6963834115c1555fea450c0ee58f26e40b508cf1c6bc69897a135a67bd"
2021-02-08T16:47:10+01:00: download chunk "/mnt/Backup/.chunks/d845/d84585c515a3ee29894c615254629b89569c2ae2457aa13c56296499971c6817"
2021-02-08T16:47:11+01:00: GET /chunk
2021-02-08T16:47:11+01:00: download chunk "/mnt/Backup/.chunks/5479/547904dccb456bb5eedba8c6692af7337c815c49c25fc7ea6d0dfbd6a36d8c51"
2021-02-08T16:47:12+01:00: GET /chunk
2021-02-08T16:47:12+01:00: download chunk "/mnt/Backup/.chunks/e7c6/e7c60accd2837c6d5e8645e5fdb3f41fe1fad3a2af74c79967b949bd05a60a26"
2021-02-08T16:47:13+01:00: GET /chunk
2021-02-08T16:47:13+01:00: download chunk "/mnt/Backup/.chunks/5bc5/5bc5fe8c28745145908f2e5af277377a2bfd970881698d27921362dc088b3511"
2021-02-08T16:47:14+01:00: GET /chunk
2021-02-08T16:47:14+01:00: download chunk "/mnt/Backup/.chunks/939c/939c8463f1fadfeb9f6dc6269ecbaac07bee61ffbc65ce0d490a4af9ccac4801"
2021-02-08T16:47:15+01:00: GET /chunk
2021-02-08T16:47:15+01:00: download chunk "/mnt/Backup/.chunks/3a1b/3a1b60198eea7fe645737d3f2d1c3fe2aef508ea5aea736aa9b7ab494620bd44"
2021-02-08T16:47:15+01:00: GET /chunk
2021-02-08T16:47:15+01:00: download chunk "/mnt/Backup/.chunks/5d47/5d479910b31d73f53956a863f5efc86132a4a18a6b82bdc6e3654fc09ea5fc8c"
2021-02-08T16:47:16+01:00: GET /chunk
2021-02-08T16:47:16+01:00: download chunk "/mnt/Backup/.chunks/d54f/d54f476378f7eed1914885d60f88c3e468d1ddb5465afb001edab583677f17fa"
2021-02-08T16:47:16+01:00: GET /chunk
2021-02-08T16:47:16+01:00: download chunk "/mnt/Backup/.chunks/862d/862de2a338cc2fbdeee376b123a8f7e47548ec392e7b642403af347898562153"
2021-02-08T16:47:17+01:00: GET /chunk
2021-02-08T16:47:17+01:00: download chunk "/mnt/Backup/.chunks/684a/684ab9b68ac3c0d1d9eae344ae7614fceca754297a29bc5318536ab8fa602d9a"
2021-02-08T16:47:17+01:00: GET /chunk
2021-02-08T16:47:17+01:00: download chunk "/mnt/Backup/.chunks/2a48/2a4812921946167b1d1fa4200b71d45701fa92292c1c168e213ea6e74bad5581"
2021-02-08T16:47:18+01:00: GET /chunk
2021-02-08T16:47:18+01:00: download chunk "/mnt/Backup/.chunks/d9b7/d9b7d75b74f48f4ac77f5598f1ebeb8aa855752eabba1c692ad6f274b388856f"
2021-02-08T16:47:18+01:00: GET /chunk
2021-02-08T16:47:18+01:00: download chunk "/mnt/Backup/.chunks/9e40/9e40d589fb98f02d186f2c7dc4d52bb0bf3b29514dbf06c8110d85d0dd31f7f1"
2021-02-08T16:47:19+01:00: GET /chunk
2021-02-08T16:47:19+01:00: download chunk "/mnt/Backup/.chunks/d4b4/d4b4a5df1e96de176e9fbba507827d4f4885395002017bc3bec1b3d5fe1977e6"
2021-02-08T16:47:19+01:00: GET /chunk
2021-02-08T16:47:19+01:00: download chunk "/mnt/Backup/.chunks/9aa5/9aa57e79f966095acb429788eecd54f027199060db70dd0ff6e6874f49486740"
2021-02-08T16:47:19+01:00: GET /chunk
2021-02-08T16:47:19+01:00: download chunk "/mnt/Backup/.chunks/8c31/8c316f9b2f3471985c17324f815bd4653270b1666af256aadc7c27cbaae03207"
2021-02-08T16:47:20+01:00: GET /chunk
2021-02-08T16:47:20+01:00: download chunk "/mnt/Backup/.chunks/1830/1830f83a44bfa3cfacfbfe0daac79d27e91063e6b738c8984ee5560e16f9b84b"
2021-02-08T16:47:20+01:00: GET /chunk
2021-02-08T16:47:20+01:00: download chunk "/mnt/Backup/.chunks/928a/928a6b8068ca3668d80692f56720e150d7cb0178923c51b5606cc7c604343aa8"
2021-02-08T16:47:21+01:00: GET /chunk
2021-02-08T16:47:21+01:00: download chunk "/mnt/Backup/.chunks/2afd/2afdbca1ec266e32e8ec36fe1da712e84d5a78e74eafff53ac107c14f430b021"
2021-02-08T16:47:21+01:00: GET /chunk
2021-02-08T16:47:21+01:00: download chunk "/mnt/Backup/.chunks/4d5a/4d5aa6a19ae72416f479729f01eb913f44986265b117b14dd3e5cf3068e2a050"
2021-02-08T16:47:21+01:00: GET /chunk
2021-02-08T16:47:21+01:00: download chunk "/mnt/Backup/.chunks/8f04/8f047b0f191ae2d6dc4a6838f3b4504dafdfb165a7af31e527b8f48ff4d8e9e9"
2021-02-08T16:47:22+01:00: GET /chunk
2021-02-08T16:47:22+01:00: download chunk "/mnt/Backup/.chunks/47f5/47f5060c2655809f38f7e4b3fb3c5064071d7a3aa6e335efbf5ea32364a554f0"
2021-02-08T16:47:22+01:00: GET /chunk
2021-02-08T16:47:22+01:00: download chunk "/mnt/Backup/.chunks/883e/883ec8843ab6c1fd62ef1a20f6485f362ac9d9faa61476f1db2371c4c6b76481"
2021-02-08T16:47:23+01:00: GET /chunk
2021-02-08T16:47:23+01:00: download chunk "/mnt/Backup/.chunks/4dcd/4dcdf5559be44289940261ff5152eae315a0ca8b8577f1ecfaa29f22fecdbee8"
2021-02-08T16:47:23+01:00: GET /chunk
2021-02-08T16:47:23+01:00: download chunk "/mnt/Backup/.chunks/47cb/47cb1a729ac5e0643e0be4d3b7fa3a5f72ef98c05a081ae75433dc38db37005f"
2021-02-08T16:47:24+01:00: GET /chunk
2021-02-08T16:47:24+01:00: download chunk "/mnt/Backup/.chunks/5491/54914143ddf5ded8ad23cb1d3692209fc5c79edb999015f6d5831f3e12833094"
2021-02-08T16:47:24+01:00: GET /chunk
2021-02-08T16:47:24+01:00: download chunk "/mnt/Backup/.chunks/18ba/18baf6f8a59e70f5ea8c98b24c761d263a491032202374f8bfd9be919436a975"
2021-02-08T16:47:25+01:00: GET /chunk
2021-02-08T16:47:25+01:00: download chunk "/mnt/Backup/.chunks/de92/de92d11590cafb3532b162acf5723c087bf851c6cd9a575c1a0a69ee3166530d"
2021-02-08T16:47:25+01:00: GET /chunk
2021-02-08T16:47:25+01:00: download chunk "/mnt/Backup/.chunks/8c8d/8c8d25bd3dbd291e3d3ba40c01ef5ac03311148e59f99526065f4c45e293bdb5"
2021-02-08T16:47:25+01:00: GET /chunk
2021-02-08T16:47:25+01:00: download chunk "/mnt/Backup/.chunks/45c8/45c8ac41a569151a6a8c87b0be9b7baecb9143a5e7b0e4e96eb59096bdeab1d6"
2021-02-08T16:47:26+01:00: GET /chunk
2021-02-08T16:47:26+01:00: download chunk "/mnt/Backup/.chunks/c295/c295ac13dfb4aee468673bbbe40ba0a7a5d02c313fb9c35e07e5b031491cbb5d"
2021-02-08T16:47:26+01:00: GET /chunk
2021-02-08T16:47:26+01:00: download chunk "/mnt/Backup/.chunks/f530/f5308201fe7492d379d06bd817b7d40f2b097ea154b0d3cdeec3cf71d1e6b563"
2021-02-08T16:47:27+01:00: GET /chunk
2021-02-08T16:47:27+01:00: download chunk "/mnt/Backup/.chunks/5dc7/5dc7382ade94fdd58f70e24cde1b463a6eedd490f35aa702b7ee150693364b17"
2021-02-08T16:47:27+01:00: GET /chunk
2021-02-08T16:47:27+01:00: download chunk "/mnt/Backup/.chunks/8b3c/8b3cb36b5bc9a64142325525d94fc18020582f394cb871d94055a7d67bd9edef"
2021-02-08T16:47:28+01:00: GET /chunk
2021-02-08T16:47:28+01:00: download chunk "/mnt/Backup/.chunks/0200/02007b3c5224fe28cb3fd97234f6ddb0a4a091d50fceca152a9b3317afcc5a40"
2021-02-08T16:47:28+01:00: GET /chunk
2021-02-08T16:47:28+01:00: download chunk "/mnt/Backup/.chunks/64e5/64e544f495f43eee0075bef602f50adf091bacb63e7eed37bd94fac22c0c936f"
2021-02-08T16:47:29+01:00: GET /chunk
2021-02-08T16:47:29+01:00: download chunk "/mnt/Backup/.chunks/fd02/fd02b7c583563c69e27da625a1cfbf8613d8136a0100ce128cb299f60b7596c6"
2021-02-08T16:47:29+01:00: GET /chunk
2021-02-08T16:47:29+01:00: download chunk "/mnt/Backup/.chunks/f6c7/f6c7a86a2d050539098c2de1532156ed3e88c9776484699458af7fbe0f31cb25"
2021-02-08T16:47:29+01:00: GET /chunk
2021-02-08T16:47:29+01:00: download chunk "/mnt/Backup/.chunks/5cc7/5cc745db8e4c4fd799ff6b247cccaab5524900099a57611fc3dbf505f45d90c2"
2021-02-08T16:47:30+01:00: GET /chunk
2021-02-08T16:47:30+01:00: download chunk "/mnt/Backup/.chunks/c2f7/c2f7bbe98531a504194e08f1f684ed67dadb770067d4c4d6d89b05435ae65372"
2021-02-08T16:47:30+01:00: GET /chunk
2021-02-08T16:47:30+01:00: download chunk "/mnt/Backup/.chunks/674d/674d90371a8593f9dd1bd73a682ea399c6fdf0780a3f53755646863dd0c1676c"
2021-02-08T16:47:31+01:00: GET /chunk
2021-02-08T16:47:31+01:00: download chunk "/mnt/Backup/.chunks/874c/874c1be5541cc8b670efbd850da3769d6b72aaa28e3fd4461ca703de62504bff"
2021-02-08T16:47:31+01:00: GET /chunk
2021-02-08T16:47:31+01:00: download chunk "/mnt/Backup/.chunks/af75/af7557b43e1c104460a9ff52d536e4b27a19738e9cebb477e14f1004557ce33d"
2021-02-08T16:47:32+01:00: GET /chunk
2021-02-08T16:47:32+01:00: download chunk "/mnt/Backup/.chunks/c7eb/c7eb0ddcb8f30dc4641b4780bd537f4c37a9d9231bb372195ffd5c002110bac1"
2021-02-08T16:47:32+01:00: GET /chunk
2021-02-08T16:47:32+01:00: download chunk "/mnt/Backup/.chunks/3aab/3aabbb4e6b602f465074d6f23eec215bcca2d116e9d288226b530b8cbe60b46e"
 
Last edited:

dcsapak

Proxmox Staff Member
Staff member
Feb 1, 2016
8,513
1,097
164
34
Vienna
network connection 1000mb down 50mb up
are you talking mbyte or mbit ? for mbit, the result does not look too bad since 3.22MiB -> 25MBit/s so roughly half

please post an iperf test in both directions and also 'proxmox-backup-client benchmark' in both directions
 

tommytom

Active Member
Aug 22, 2015
32
5
28
are you talking mbyte or mbit ? for mbit, the result does not look too bad since 3.22MiB -> 25MBit/s so roughly half

please post an iperf test in both directions and also 'proxmox-backup-client benchmark' in both directions
to clarify: i have 1 gigabit downstream and 50mbit Upstream on my home sync PBS site which produces slow speed.

the main backup-server server is connected with 10gigabit to the network 30-50ms ping time away

as i wrote.. sometimes i had even 40megabyte download speed. before it dropped on a very huge vm with 1.2Tb disk size.
 
Last edited:

dcsapak

Proxmox Staff Member
Staff member
Feb 1, 2016
8,513
1,097
164
34
Vienna
to clarify: i have 1 gigabit downstream and 50mbit Upstream on my home sync PBS site which produces slow speed.

the main backup-server server is connected with 10gigabit to the network 30-50ms ping time away
it is still not clear in which direction you did benchmark, also does the backup-server really have a 10gbit uplink?

as i wrote.. sometimes i had even 40megabyte download speed. before it dropped on a very huge vm with 1.2Tb disk size.
this also very much depends on the storage + caches, e.g. if you have 'slow' mechanical disks and the chunks are not in the memory cache, speeds will be much lower
this: https://git.proxmox.com/?p=proxmox-...0f58d259606b643df542e2664b08009fe;hb=HEAD#l64
has some napkin math calculations for how much speed you can get, but of course higher latency links will be detrimental to speed
 

tommytom

Active Member
Aug 22, 2015
32
5
28
False alarm!
It's because of our network connection. We have a 1GigaBit line with Vodafone as a business customer. Unbelievably, they throttle the line down to 10mbit during the day. Shortly after midnight, everything works at full speed. Then you also get a good performance with the PBS as expected.

I cannot recommend Vodafone in Germany. We will now probably get a line with Colt.

Thanks for your great work !
 

dbayer

Active Member
Apr 15, 2016
63
8
28
52
False alarm!
It's because of our network connection. We have a 1GigaBit line with Vodafone as a business customer. Unbelievably, they throttle the line down to 10mbit during the day. Shortly after midnight, everything works at full speed. Then you also get a good performance with the PBS as expected.

I cannot recommend Vodafone in Germany. We will now probably get a line with Colt.

Thanks for your great work !
How did you discover they were throttling?
 
  • Like
Reactions: DerDanilo

tommytom

Active Member
Aug 22, 2015
32
5
28
because i checked the line frequently and found out, that single line connections like with wget a throttled in the evening hours and get back to fullspeed short after 00:00 am - every day... from up to 50mbyte/s they throttle it to 500kb-800kb/s
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!