limit concurrent tasks

Feb 20, 2021
73
6
8
Is it possible to limit concurrent tasks?

We use a large spinning disk as archive and it becomes really slow when the prune / gc / sync / verify jobs run at the same time.

It would be nice when these tasks can be run sequentially: if a task is already running, wait until its finished and then start the next one.
 

RolandK

Member
Mar 5, 2019
162
19
23
49
yes, if i backup a cluster, i see a very large number of concurrent verify tasks when i enable "Verify new backups immediately after completion" option, so many concurrent tasks want to get their share on cpu and Io.

i cannot guess that this is effective with a large number of tasks
 
Feb 20, 2021
73
6
8
Code:
 i cannot guess that this is effective with a large number of tasks

thats why the limit should be configureable. for spinning disks its mostly faster when only one concurrent task is running. The more tasks you start in parallel, the slower it gets because spinning disks are very slow when it goes to random access. Linear access is much faster.

On a spinning disk it would be much more effective having only one task as the bottleneck mostly is the disk.

i guess its best to write a script for that which invokes the tasks via proxmox-backup-manager one by one.
 
Last edited:

sukerman

Member
Aug 29, 2019
50
4
13
50
I'm having the same issue, concurrent tasks causes IO delay to hit 70% then its starts crawling
 

itNGO

Well-Known Member
Jun 12, 2020
526
114
48
44
Germany
it-ngo.com
Still not possible in 2.2, but you can create your jobs on PVE that they have enough time to finish and then schedule the next one.
A bit of manual work, but it can work....
 

k-123

Member
Mar 3, 2017
16
0
21
38
when our pve nodes expanded from 3 to 5, the IO issue on target pbs goes much worse. (sometimes a small mount of VMs backup will failed)
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!