Remote backup error - broken pipe

[B3rt_PX]

Hi, I have this problem with a my customer's backups.
On my customer's network I installed a PVE with inside installed one virtual machine.
I connected one Proxmox backup server storage (a remote datastore into a remote PBS, connected by IPSEC VPN) and configured a schedule backup on it. The backup starts successfully for the first 30/40 minutes, after this time the process failed with the error "broken pipe".
After 3 weekes, I tryed to install and configure a local PBS on a customer's host, and after I configured a remote sync on a remote PBS host.
In this way the local backup is sucessfull and remote sync is sucessfull, although the process lasts 30/40 minutes.
That I don't understand is the difference between these 2 procedures (directly remote backup through Proxmox Backup Server storage and the procedure which consist in local backup + remote sync).
And why does one procedure fail while the other is performed correctly?

I attach the log error registered at the moment of failure backup:

INFO: started backup task 'a998d70b-0867-4864-9986-6c5c386f6262'
INFO: resuming VM again
INFO: scsi0: dirty-bitmap status: existing bitmap was invalid and has been cleared
INFO: 0% (652.0 MiB of 700.0 GiB) in 3s, read: 217.3 MiB/s, write: 18.7 MiB/s
INFO: 1% (7.2 GiB of 700.0 GiB) in 4m 25s, read: 25.8 MiB/s, write: 8.2 MiB/s
INFO: 2% (14.1 GiB of 700.0 GiB) in 6m 50s, read: 48.4 MiB/s, write: 14.2 MiB/s
INFO: 3% (21.2 GiB of 700.0 GiB) in 12m 48s, read: 20.4 MiB/s, write: 4.6 MiB/s
INFO: 4% (28.1 GiB of 700.0 GiB) in 18m 32s, read: 20.3 MiB/s, write: 8.2 MiB/s
INFO: 5% (35.3 GiB of 700.0 GiB) in 23m 46s, read: 23.6 MiB/s, write: 8.0 MiB/s
INFO: 5% (35.3 GiB of 700.0 GiB) in 39m 47s, read: 0 B/s, write: 0 B/s
ERROR: backup write data failed: command error: write_data upload error: pipelined request failed: broken pipe
INFO: aborting backup job
INFO: resuming VM again
ERROR: Backup of VM 100 failed - backup write data failed: command error: write_data upload error: pipelined request failed: broken pipe
INFO: Failed at 2022-04-19 22:39:52
INFO: Backup job finished with errors
TASK ERROR: job errors

Can someone help me?
The PBS's version is 2.1.5

Thanks

 

[fabian]

a backup happens by the PVE host connecting to the PBS host, upgrading to HTTP2 and then sending chunks with data and other things over this connection. the logs would indicate that something interrupts the connection.

syncing happens in the other direction - the PBS system where the sync task runs connects to the remote PBS and pulls the chunks and metadata. for a single snapshot this also uses a single HTTP2 connection. why it doesn't get interrupted when syncing I can't really tell - maybe some firewall/router/.. treats incoming vs. outgoing connections differently? syncing might also be faster depending on which storages are involved.

 

[B3rt_PX]

a backup happens by the PVE host connecting to the PBS host, upgrading to HTTP2 and then sending chunks with data and other things over this connection. the logs would indicate that something interrupts the connection.

syncing happens in the other direction - the PBS system where the sync task runs connects to the remote PBS and pulls the chunks and metadata. for a single snapshot this also uses a single HTTP2 connection. why it doesn't get interrupted when syncing I can't really tell - maybe some firewall/router/.. treats incoming vs. outgoing connections differently? syncing might also be faster depending on which storages are involved.

No the incoming/outgoing connection is the same, but I can't explain myself why using remote sync the process doesn't interrupt/fail, unlike remote backup. With the same logical it should also stop with remote backup, but it's not.

 

[fabian]

no, if you have the following setup

location 1:
PVE, local PBS

location 2:
remote PBS

doing a backup from PVE to remote PBS and doing a sync from local PBS to remote PBS will not be the same direction - in one case PVE (location 1) opens the connection, in the other the remote PBS (location 2)