Slow backup verification on S3 datastore

[adhisimon]

I'm using S3 endpoint datastore (wasabi) on Proxmox Backup Server. Backup is fast enough (although sometimes failed, but I can accept it).

My problem is it was so slow when doing backup verification. Looks like PBS only use one thread when doing verification. I see it's only one socket established to wasabi s3 endpoint.

Is it my assumption on single thread is true? Is there any plan to make verification running on multiple thread, so it can download chunks from s3 storage in parallel?

 

[p-user]

You are correct, verification on an S3 backend is slow. So I disabled the option "Verify New Snapshots" in the Options tab of the S3 datastore.
This option makes a verification run right after a backup is made.

I have configured a separate verification job for this, and that runs on a more convenient time than after every backup automatically.

If it is single threaded I have no idea. But the solution works for me.

 

[adhisimon]

You are correct, verification on an S3 backend is slow. So I disabled the option "Verify New Snapshots" in the Options tab of the S3 datastore.
This option makes a verification run right after a backup is made.

I have configured a separate verification job for this, and that runs on a more convenient time than after every backup automatically.

If it is single threaded I have no idea. But the solution works for me.

The opposite with my workaround. Separate verification job will run verification sequentially on scheduled time. To maximize bandwidth, I enabled "Verify New Snapshots", this will make verification of each snapshot run in parallel.