Backup to multiple backup storages

[alexx]

Hi,

I'm currently running 25 Proxmox nodes in a cluster (ceph storage backend) which are backed up to a Proxmox backup server every night. I set up separate backup jobs for each host starting every 30min (average backup time of a host is 20min). Because of space/load issues on the backup server I want to add another one. How would one organize this?

If I backup half of the nodes to each backup server both backups servers will fill up with all VMs as VMs are moved over time.

If I create a backup job including all nodes for each backup storage/server , selecting half of the VMs in each jobs, the backup would run on multiple hosts in parallel which also isn't desired. (Its quite unfortunate that there isn't the possibility to limit the parallel running backup jobs for the jobs on all hosts).

A solution would be to setup a singe backup job for every VM, but this is quite error prone and doesn't scales well.

Any ideas? Whats the preferred solution for backup of multiple nodes to multiple backup storages?

 

[UdoB]

Whats the preferred solution for backup of multiple nodes to multiple backup storages?

I have no idea, the following is just my own idea:


Your old PBS is fast (enough)? --> Build a new PBS with larger capacity, possibly slower (speed is welcome but not the priority).

Your old PBS is slow? --> Build a new PBS with NVMe/Sata only. The capacity may be lower.


My point is: have one fast but relatively small PBS as the primary / daily system. This also allows fast restore. Have a second one with much larger capacity, possibly slower and resulting in a slow restore, which hopefully should never be necessary ;-)

Setup a sync/pull job to fetch backups from the small PBS to the larger one.

Do NOT backup to PBS1 today and then to PBS2 tomorrow - as the "Dirty Bitmap" will get invalidated every time you switch the destination.

Set a much shorter retention on the first PBS. Let the second PBS keep backups "forever", depending on the actual available size or actual requirements.


You might also search for a tutorial on how restrict access to (at least) the second one to make it malware/ransomware proof. Nobody in the cluster should have the right to delete backups. Never.

You did not tell us which type of storage is used. Make sure to implement a supported one, in any case avoid "rotating rust only".