Slow backups sabotaging VMs

[itkfm]

Hi,

We have seen multiple cases of oddly slow backups that end up sabotaging the VMs (stalling them to death).
It happens across various constellations, locations and PBS versions. It’s not exactly deterministic, but there’s one server where it happened twice successively.

I’ve not been able to spot anything interesting in the logs so far.
All I can tell is that the backup transfer speed seems to go south out of nowhere.

Any ideas what to look into?

 

[itkfm]

To clarify:
The issue is not that a slow backup stalles a VM. It’s rather that sometimes backups take forever for seemingly no reason.

Involved storage: Local ZFS mirrors.

 

[bbgeek17]

Any ideas what to look into?

I would think the steps are the same as troubleshooting any generic slow application whose job is to transfer data:
- record/analyze CPU load on both side
- record/analyze disk load
- record/analyze network load, errors

You may want to install something like Grafana to point at PVE/PBS so you can correlate pretty graphs and have continues monitoring.
There is nothing you can do without first collecting the data.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[_gabriel]

is PBS over WAN ?
Too slow PBS can impact VM because when data change in VM, old data is first sent to PBS then VM wait PBS to finish, only after it's done, new data in VM can be effective written/modified.
There is fleecing option to mitigate this case.
(sorry for my bad english wording)