Second backup server best practices?

[godzilla]

Hi all,

I'm currently running one PBS for my cluster, which stores data on an NFS share backed by an enterprise-grade QNAP storage.

Everything went fine until about two weeks ago, when I noticed some scheduled backups were starting to file sometimes. Also, I see that when I browse VM backups on the PBS itself it takes a very long time.

So I was thinking about adding a second backup server, but I understood that they can't use the same datastore. What's your advice in cases like these?

Thank you!

 

[Dunuin]

Setup a second PBS, preferably offsite, and then use the sync task to let the second PBS pull the backups from the first PBS. That way both PBS store the same backups and you only need to do the backups once to the first PBS.

I guess that QNAP is using HDDs? You really shouldn't use HDDs with PBS. It's neither designed for HDDs nor for network shares, as it needs high IOPS performance. Local SSDs would be ideal.

 

[LnxBil]

enterprise-grade QNAP storage

Could you please elaborate what type this is? In their ZFS-based system, there may also be the option to replicate the storage to an off-site location.

 

[godzilla]

Thanks for your replies.
Sorry, maybe I explained myself badly. I don't need replication or IOPS, I need more processing performance.

Sure I could upgrade the current server in terms of CPU and RAM but I'd rather add another server and make them work in parallel if possible.

 

[LnxBil]

I don't need replication or IOPS, I need more processing performance.

So, you need more IOPS ... data does not always lays around sequentially on the disk...

 

[godzilla]

So, you need more IOPS ... data does not always lays around sequentially on the disk...

I have no bandwidth or latency issues on the storage server. What makes you think that?

 

[Dunuin]

I have no bandwidth or latency issues on the storage server. What makes you think that?

Running a GC job on the PBS doesn`t need a lot of CPU power or RAM but lots of IOPS performance, as it needs to read and write the atime of millions of chunk files. Here you are usually bottlenecked by the IOPS performance of the HDDs. How long does your GC need to finish?

What "IO delay" is PBS reporting when doing backups/restores/verifies? It should be really low, otherwise the disks are bottlenecking.
Biggest CPU performance should be needed when doing verify jobs.

 

[godzilla]

Hi @Dunuin, thanks for your reply.

GC jobs on the main datastore which uses about 17TB used to take ~2 days. Anyway, following @LnxBil 's advice yesterday I added some SSD to the QNAP and activated the autotiering feature, and boom! Latest GC took only 4 hours.

I don't see any "IO delay" in the logs, but it looks like every task is now much quicker to execute (it could also be due to the fact that I rebooted both the QNAP and the PBS). So perhaps I "fixed" the issue this way?

 

[LnxBil]

Anyway, following @LnxBil 's advice yesterday I added some SSD to the QNAP and activated the autotiering feature, and boom! Latest GC took only 4 hours.

Great!