Hi, in case this is helpful for other people, (or for me to find this hint in future
)
I've got a basic client site config with
Proxmox server onsite at office
PBS server onsite at office
PBS server offsite in data centre in nearby city / for offsite data sync from PBS_onsite replication of backups
The challenge is that the office has a pretty mediocre ISP service (200meg down Cable modem with 10meg up as bandwidth)
which means at 100% throughput for upload from office>datacentre PBS data sync, it will use all the upload bandwidth
which is ~1Mb/second throughput upload on the 10Mbit connection
So, with some google fun digging, I learned about 'wondershaper' which is a small script available on debian
and I've setup a trivial cron config on my onsite PBS host
so that every day Monday-Friday at 7am it will throttle upload bandwidth to 4Mb or ~400Kb/s upload speed
and then Monday-Friday at 8pm it will relax things to allow full speed upload at ~10-11Mb or ~1Mb/s upload speed
So far this seems to do the trick, if I've got long-running PBS jobs that span multiple days, they don't care if things are throttled at the client site office side of things. The connections remain active-ongoing, they are just being throttled at the linux stack side of things on the client-site PBS box.
For clarity - the intent of this setup, is to allow staff at the office to work M-F during 'normal hours' and have limited impact on the day-to-day work people are doing. Most of what the staff want is download bandwidth constrained with some modest upload requirements (intermittent zoom - teams calls) plus of course any upload required for download to work smoothly (ie, ACK). The secondary goal here is to allow PBS backup jobs to just keep running if required for >24 hours and allow things to gently throttle down and up / so that we get better PBS bandwidth use outside of core office hours. Right now the office onsite PBS backups are 'not up to date' so need multiple days to get caught up. The expectation is that - with full weekend of full bandwidth, plus all week with ~night time full bandwidth and dayime ~half bandwidth - the PBS offsite server will get caught up at least once per week. But it might be playing catch-up for many other days of the week. Which is tolerable.
So. lots of fun. In perfect world we just get a bigger pipe outbound at this site, but that isn't going to happen anytime soon, so here we are.
So, in case this is helpful here is what my config looks like for reference -pasted below.
-Tim
Code:
CRON HINT
========================================================
Min Hour Day Mon Weekday
* * * * * command to be executed
┬ ┬ ┬ ┬ ┬
│ │ │ │ └─ Weekday (0=Sun .. 6=Sat)
│ │ │ └────── Month (1..12)
│ │ └─────────── Day (1..31)
│ └──────────────── Hour (0..23)
└───────────────────── Minute (0..59)
BASELINE SPEED OBSERVED on the onsite PBS box with no throttle and active PBS job uploading:
============================================================================================
root@pbs:~# ifstat
enp3s0 enp2s0
KB/s in KB/s out KB/s in KB/s out
0.00 0.00 22.30 1156.94
0.06 0.00 27.77 1096.26
CRONTAB ENTRIES FOR THROTTLE CONTROLS AUTOMATION:
=================================================
### MON-FRI we INCREASE speed for upload to ~11MB at 8pm
0 20 * * 1,2,3,4,5 wondershaper enp2s0 110000 11000
## MON FRI we drop speed upload to ~4MB at 7am
0 7 * * 1,2,3,4,5 wondershaper enp2s0 110000 4096
ONCE THROTTLE IS IN PLACE WE SEE BANDWIDTH
===========================================
root@pbs:~# ifstat
enp3s0 enp2s0
KB/s in KB/s out KB/s in KB/s out
0.06 0.00 11.81 533.66
0.00 0.00 11.91 539.17
