sync group filed - connection closed because of a broken pipe

[herzkerl]

Starting a while ago we've had some issues with syncing some backups to our offsite PBS.
Upgrading to v3 didn't change that, unfortunately.

The task would fail with the following message:

 sync group vm/208 failed - connection closed because of a broken pipe

That only happens for about 50% of our VMs—what's the best way to troubleshoot this?