PBS and pools of VMs

[Melcosoft]

Hi every one. Have a question about backup process for pool of VMs.

I have 20 PM servers and on each one I have at least one critical VM.
I created a pool called "Critical" and add from all 20 PM servers at least 1 VM to that pool.
That means that I have 20 VMs on pool called Critical.

Then I created a scheduled backup for that "Critical" pool every 4 hours.

When I'm running that Scheduled Backup for Critical - it's running 20 backups on a same moment immediately because it's a pool of 20 different servers.

Question:
Is there any way to make Scheduled Backup for pool to run only one backup per time like if I'm doing that for a host?

because when I'm running a pool backup - I have 20 tasks immediately running and my PBS and storage going overloaded.
So what I need is have only one backup process on pool per moment until all VMs from pool will be done.

 

[tcabernoch]

Mine doesn't do that.
You should only get one backup running per PVE host at a time.
If you are getting multiple backup threads running on a single host at the same time, I want to know how.
It's more likely that for say a cluster of 5 hosts, you get 5 backups running at once.

EDIT ... I may have miss-read your post. You have a 20-member cluster? That's not typically recommended, even on a fast network. It is done, but on higher end gear than most would install PVE on. If you have a 20 member cluster, and you are running a backup job that spans VMs across the entire set of servers, then a cascade of 20 simultaneous backup jobs is by design, and cannot be changed that I know of.
If this is the case, you should consider constraining your backups by Server as well as Pool ... and follow the staggering directions below.

As for how to stagger them ...
If you got as far as Pools, then you are already well familiar with the Proxmox backup group management tools at your disposal.
There's only one. Pools. So, use it. Cut your Pool in half.
Instead of a large Pool called Critical, create a Pool half the size called Critical-early and another one called Critical-later. Schedule one at 2 hours and the second at 4 hours.
As steady and even performance is your goal, be mindful of the VMs' host placement so that your backups balance across your hosts somewhat evenly.

...
Bonus stuff ...
Here. I use Pools for Backup.
I explored how to automatically add new VMs to a Pool, so they would automatically be backed up.
Wrote a script. You might eventually find it useful.

Thread 'Automated Backup Management in Proxmox via Pools'

Dec 5, 2024

I would like to automatically add new VMs to a Pool.

I manage backups via Pool membership.
I would like to automatically backup new VMs.
So it would be great if I could automatically stick em in a "New VMs" Pool.

How would you do that? I've heard about hook scripts. I could imagine much too complex methods involving shell scripts.

NOTE ... Scroll down. I built it all, screenshots and stuff ...

• tcabernoch
• Replies: 14
• Forum: Proxmox Backup: Installation and configuration

 

[Melcosoft]

Yeah. That's what I was trying to avoid.
If there is no way to run pool backups with only one concurrent backup - then I will create 20 pools call Critical for each host.
Only that can fix my issue for now...

 

[tcabernoch]

To the best of my understanding, there is no global setting that will allow you to reduce the amount of concurrent backups.
The staff would undoubtedly direct you to the bug tracker to make a feature request. There probably already is one.

Again, to the best of my understanding of your configuration, you'll need to use Server constraints in addition to Pools in order to accomplish what you describe.

 

[Melcosoft]

Thx again. I'm positive of waiting for that new feature and there is no critical rush for me right now.
It's just a bit mess to have so many pools or backup scheduled tasks. But still possible to fix for now.

And yes. I have 20 PM servers. I have 20 other ones on VMware which is I'm currently migrating to PM. I have one PBS with 6 NFS storages (thru QNAP).
Ones I finish migration - I want to make it visibly cool and easy for understanding what I did for and other new IT persons in my company so I'm trying to make pool's and backup process easy and fast.