Cluster and backups

[john.wayne]

Hello

We got a ProxmoxVE cluster having 20 nodes and are "trying" to backup all VM's overnight.

Currently it will start 20 backup tasks (1 per node) which will result in very high I/O delays as the destination NAS won't be able to handle the load.

If we limit the bandwith for the backups, some of the backups won't complete after 3 days, not really ideal.

* Are there any best practices for backups having lot of nodes?
* Are there other backup options if we would use ceph-storage?
* Could differential backups (github.com ayufan/pve-patches) be a solution for this case?

Thank you

 

[LnxBil]

* Are there any best practices for backups having lot of nodes?

Yes, use a decent backup system. It seems that your bottleneck is the NAS. Replace it with a much bigger machine. I hope you also have at least 10 GBE for that. Writing 10 GBE is write throughput of over 1 GiB per second, so you really need a lot of disk spindels or pure SSD for a concurrent stream of 20 nodes.

* Could differential backups (github.com ayufan/pve-patches) be a solution for this case?

No way! This not only takes much longer, it also reads the data beforehand, so you're going to increate the overall I/Os tremendously.

 

[guletz]

In your situation (NAS bottleneck ) maybe you can use another approach. Backup your VM/CT from x1 node to x2 node, x2 to x3, ...and so on(short term backup). Then you can copy your short term backup to the nfs, as your performance is acceptable. Yes it could take a lot of time(maybe weeks until you have let say all backups older then 2 weeks on the nas), but I think is the best you can do in your case.