PBS4 S3 datastore - sync speed

T

trevorludgate

Active Member
Jul 7, 2020
4
0
41
29
Hi,

I'm curious what upload speeds most of you are getting on S3 syncs. I've been able to sustain upload speeds of around 3Gbit/s when uploading with rclone, but I can only reach speeds of around 50Mbit/s from PBS. Obviously there's quite a few variables at play (number and sizes of files, compression...), but I'm wondering if this is something that I can tune to increase the upload performance. I've tried this on 2 different S3 endpoints (Backblaze and Mega S4) and receive the same speeds. The test with rclone is on the same underlying storage as the backup source datastore. The cache drive has 1.5TB free and is on NVME storage.
The PBS server has 4 cores and 8GB of memory, both at relatively low usage.

This isn't the end of the world as the incremental syncs won't take too long, but it would be nice to be able to speed it up and I'm curious if anyone else has noticed the same behaviour.

Thanks in advance!

Trevor
 
I'm in SoCal, and use the US West 004 Backblaze B2 site. This morning on my first/fresh full backup to B2 I was getting 94MB/s (yes BYTES) on my 1 Gbps symmetrical internet connection. 110GB of data uploaded in about 20 minutes. I was watching my firewall stats the whole time and it was basically pegged at 940 Mbps the whole time.
 
  • Like
Reactions: trevorludgate
Interesting. Out of curiosity, are you running PBS on bare metal? ZFS?
I noticed that I'm getting high read latency (~20-30ms) on the underlying disks during the sync operation when I don't see that behaviour at all when copying files with rclone or when syncing to a remote PBS site.
When running an S3 sync, the read speeds on the HDD array are very slow (only a couple of MB/s on each drive, the array is 6x mirrors of 2 drives). The write speeds on the nvme drives reflect this (as they're being used as the cache in the S3 datastore).
As a test, I set up a second datastore on my nvme array. When syncing the same backups from the HDD array to the nvme array, I'm seeing speeds of around 500MB/s, which is more inline with what I was seeing during my clone tests. I'm going to let this sync run for a bit, and then try to upload the backups directly from the nvme (with the cache on the same array) to isolate any issues with the HDD array.
 
You must log in or register to reply here.
Top Bottom