Reduce backup I/O but only for one host

[mixman68]

Hello,

We have a cluster with 4 PVE and 1 QDevice,

One of our PVE is old and has spin HDD, how I can apply backup bandwidth limitation only on this host,

The option is on datacenter options so I think it was applied to entire cluster

 

[waltar]

Perhaps "man ionice" can help you to start your binary with a script doing ionice ... backup-bin or you could change the running process with it.
But don't really understand your problem as you mean that your old pve has slow I/O which is slowing your backup already ...

 

[mixman68]

Perhaps "man ionice" can help you to start your binary with a script doing ionice ... backup-bin or you could change the running process with it.
But don't really understand your problem as you mean that your old pve has slow I/O which is slowing your backup already ...

My problem is the backup use 100% of io (I see on Iotop) and load average go over 50 for a 8 thread cpu, vm are unresponsive
When I put limit of 10 Mo/s on the cluster, my backup take only 2x more time but load average doesn't go over 10

I use the integrated proxmox solution to backup

 

[waltar]

First "100% i/o in iotop" means that process|core which initializes the I/O wait's and could do it's work which then let rise the load you see.
Did your iotop show "SWAPIN IO>" also otherwise do "echo 1 >/proc/sys/kernel/task_delayacct" before starting iotop
then the sorting is default onto "IO>" instead onto "DISK WRITE" :
Total DISK READ: 0.00 B/s | Total DISK WRITE: 0.00 B/s
Current DISK READ: 0.00 B/s | Current DISK WRITE: 0.00 B/s
TID PRIO USER DISK READ DISK WRITE "SWAPIN IO>" COMMAND
1310522 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.04 % [kworker/u98:0-flush-252:1]
...
"When I put limit of 10 Mo/s on the cluster" - mmh, where's that, we never try to slow down anythink ... ?!
"my backup take only 2x more time but load average doesn't go over 10" - looks like you solved your problem yourself with limiting ??

 

[mixman68]

Yes I did this.

But I solved partially cuz the limit apply to all the cluster, I want to set this only on one host

Do you know how tweak vzdump.conf for this ?

Thanks

 

[waltar]

I'm sorry, no. From shell/cron of that old host you also could re-nice your running backup pid (nice 19 <pid>), it's outside of proxmox but that bring down the load on that host.

 

[UdoB]

The option is on datacenter options so I think it was applied to entire cluster

Every single backup job has an independent "Bandwidth Limit:" on the "Advanced"-Tab. Please see https://pve.proxmox.com/pve-docs/chapter-vzdump.html#vzdump_jobs

I am not sure if this limits the Ethernet line speed or the read speed from the source HDD. Just try and see what happens im you enter "1" MiB/s...

Good luck.

 

[waltar]

Mmh, the really right way would be to resolve your I/O problem, if I/O wait is 0.0 your load max out at 8 with 8 cores having 100%, when have less even load goes even deeper. You should consider that for your future changes. If your storage is raidz"x" change to "mirror" and your problem might high probable solved and even your vm's run better.