Scheduling Backup and System Maint w/PBS

[tcabernoch]

Here are the things that our PBS systems do.

• Backup Jobs
• Local Sync Jobs (Primary and Secondary PBS)
• Site to Site Sync Jobs (Both directions)
• Pruning Jobs
• Garbage Collection Jobs
• Verify Jobs

Some of those things need to run after each other.

• Sync Jobs run after Backup Jobs, and Site to Site Jobs run after those.
• GC runs after Pruning.

But then there's Verify, and that can take all darn day. It seems the system throttles this activity when needed, but I'm sure it has an impact.

And Site to Site Sync Jobs have no priority, but they can't run at the same time as each other. One direction at a time.

Anyway, I need to sort all these out and schedule them.
I guess I'm gonna just put them all in a spreadsheet and figure it out.
Got any suggestions?

 

[t.lamprecht]

You normally do not need to schedule them so exactly that you really need a spreadsheet to figure it out.
And all jobs handle running in parallel just fine in general, albeit verify and GC naturally have a relatively big (read) IO impact, and might cause some IO pressure for other jobs.

Anyhow, what makes sense in general is to schedule the GC after a prune, since running two GCs back-to-back without pruning (or manual backup snapshot deletion) in between won't free up any space. But as pruning is pretty fast you can just do GC like half an hour later and be done (or check your average job duration for the odd case that prune jobs are really slow in your setup).

Sync jobs will pick up whatever is ready, so if you always want a very up-to-date state then you can just increase the frequency of syncs, but otherwise you do not need to schedule them closely.

Verify can indeed take quite a long time, and while it's definitively good to verify backups occasionally to catch some storage level error relatively early, it's still not something that one needs to trigger very often. Doing it after pruning might make sense as that means there's potentially less to verify.
Syncing your PBS to a separate server is IMO often a better hedge against storage fallout, and then a weekly verification might be well enough – but it really depends on your exact use case and requirements.

 

[tcabernoch]

Hey, thank you! Alright, that seems like good general advice as I take this on.
There's not a lot of talk about syncing. Very helpful of you here.