Limit simultaneous backup jobs

Afox

Active Member
Dec 18, 2014
235
11
38
Hello,

on my node it is possible to start multiple backup jobs at the same time. As this often results in conflicts I want to ask how I can limit the simultaneous backup jobs?

Thanks in advance.

Best regards,

Afox
 
Last edited:

fiona

Proxmox Staff Member
Staff member
Aug 1, 2019
2,440
484
88
Hi,
if you select multiple VM IDs for a backup job, the backup of each VM/CT will be executed sequentially.
 

Afox

Active Member
Dec 18, 2014
235
11
38
Hello, thank you for the answer but that is not my problem.

The problem is running 2 backup jobs at the same time what is possible recently and wasn´t possible before.

For example one backup job is starting at 12 o'clock and another at 13 o'clock. If the first backup job hasn´t finished until then the second will start anyway (what I don´t want). I want that the second job waits until the first has finished.
 

fiona

Proxmox Staff Member
Staff member
Aug 1, 2019
2,440
484
88
Just to be clear: you have configured an hourly vzdump cron job that (sometimes) takes more than an hour to complete? Or is the job starting at 13 o'clock a different one?
 

Afox

Active Member
Dec 18, 2014
235
11
38
The second job is a different backup job but on the same node.
 

fiona

Proxmox Staff Member
Staff member
Aug 1, 2019
2,440
484
88
Are you sure the behavior was different previously (which version)? I think this would require a new feature, i.e. limiting the number of workers per node for a given task type, or alternatively having a queue of backup jobs (and not timer/cron-based).

What exactly are the problems with the simultaneous jobs? Is scheduling the second job later, or using the bwlimit feature an option?
 

Afox

Active Member
Dec 18, 2014
235
11
38
yes, i´m sure. there was a global lock when the first job was running and you couldn´t run a second backup job (manual or cron). It was stating a "wait" status until the running job was finished and then the second job started.

The problem with the current behaviour is that when the second job runs at the same time it can try to backup a VM that is currently backed up by the first job resulting in an error and skipping that VM.
 

fiona

Proxmox Staff Member
Staff member
Aug 1, 2019
2,440
484
88
yes, i´m sure. there was a global lock when the first job was running and you couldn´t run a second backup job (manual or cron). It was stating a "wait" status until the running job was finished and then the second job started.
That is done for pve-zsync, but not for vzdump AFAICT. Which PVE version would we be talking about?

The problem with the current behaviour is that when the second job runs at the same time it can try to backup a VM that is currently backed up by the first job resulting in an error and skipping that VM.
What is the use case? I.e. why back up the same VM with different job configurations so close together in time?
 

Afox

Active Member
Dec 18, 2014
235
11
38
PVE 6.3-6
One is a weekly job and another a daily with different compression. Backups should run when the load is low so the timeframe is not very huge.
 

fiona

Proxmox Staff Member
Staff member
Aug 1, 2019
2,440
484
88
Thanks for the report! It indeed seems to be a regression. We'll come up with a fix and restore the previous locking and lockwait behavior.

EDIT: A fix has been applied in git and will be available in future packages.
 
Last edited:
  • Like
Reactions: Afox

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!