Storage Lock timeouts when deleting multiple VMs, no errors

Big4SMK

Renowned Member
Jun 7, 2017
28
2
68
42
I'm on PVE 8.3.4 with a raid6 array of 10krpm disks backing local storage, when facing the following issue. I try to delete ten VMs through the API at (about) the same time, unfortunately some of the VMs face a timeout as they can't lock /var/lock/pve-manager/pve-storage-local. I started all tasks at 10:00:10, the first four complete successfully at 10:00:11, :12, :13 and :18 and the rest of my destroy jobs all ended at 10:00:20 with a timeout, so it seems that the timeout is only 10 seconds. The failed tasks still return "TASK OK", even though they leave disk images behind.

I have two questions:

1) Why do the destroy tasks still return "TASK OK", even though the disk couldn't be removed. Is there any way for me to catch this error through API without downloading and parsing the log messages?
2) Can I influence the timeout, 10 seconds seems fairly aggressive for deleting VMs in bulk.