PBS S3 datasotre sync job, "SlowDown" response handling.

rahman

Renowned Member
Nov 1, 2010
79
4
73
Hi,

We are trying new S3 support for syncing local datastore to S3 backend (CEPH cluster/Rados GW). The problem is some times S3 endpoint respond with HTTP 503 "SlowDown" and PBS S3 client immediately retry that chunk. But when S3 endpoint respond with multiple SlowDown's PBS S3 client does not back-off and retry later, it just gives up trying to send that VM snapshot and continue with the next snapshot.

I tried the new rate limiting functions (the ones on WEB GUI) but it does not solve the issue because it is not about rate limiting trafic MiB. I tried "put-rate-limit" in cfg file. When I use 1 request per second, it works without any 503 SlowDown errors. 3 request per minutes also seems to work but 5 and above causes SlowDown responses and if the VM snaphot is big (1 TB in our case), S3 client can not complete sending all chunks.

Is there any way to improve this? Hard limiting put requests per second is not an option because it's too slow. Does PBS S3 client has some kind of back-off algorithm?

Regards,

Rahman

Code:
2025-11-28T08:35:59+03:00: Starting datastore sync job '-:default:ulaknetfkm:default:s-4d4c0e22-da65'
2025-11-28T08:35:59+03:00: sync datastore 'ulaknetfkm' from 'default'
2025-11-28T08:35:59+03:00: ----
2025-11-28T08:35:59+03:00: Syncing datastore 'default', namespace 'default' into datastore 'ulaknetfkm', namespace 'default'
2025-11-28T08:35:59+03:00: found 127 groups to sync (out of 127 total)
2025-11-28T08:35:59+03:00: skipped: 1 snapshot(s) (2024-05-04T05:00:08Z) - older than the newest snapshot present on sync target
2025-11-28T08:35:59+03:00: re-sync snapshot vm/103/2024-06-01T05:00:05Z
2025-11-28T08:35:59+03:00: no data changes
2025-11-28T08:35:59+03:00: percentage done: 0.79% (1/127 groups)
2025-11-28T08:35:59+03:00: re-sync snapshot vm/106/2024-01-06T05:00:17Z
2025-11-28T08:35:59+03:00: no data changes
2025-11-28T08:35:59+03:00: percentage done: 1.57% (2/127 groups)
2025-11-28T08:35:59+03:00: skipped: 1 snapshot(s) (2025-10-04T05:00:01Z) - older than the newest snapshot present on sync target
2025-11-28T08:35:59+03:00: re-sync snapshot vm/107/2025-11-01T04:00:04Z
2025-11-28T08:35:59+03:00: no data changes
2025-11-28T08:35:59+03:00: percentage done: 2.36% (3/127 groups)
2025-11-28T08:35:59+03:00: skipped: 1 snapshot(s) (2025-11-16T04:00:04Z) - older than the newest snapshot present on sync target
2025-11-28T08:35:59+03:00: re-sync snapshot vm/109/2025-11-23T04:00:10Z
2025-11-28T08:35:59+03:00: no data changes
2025-11-28T08:35:59+03:00: percentage done: 3.15% (4/127 groups)
2025-11-28T08:35:59+03:00: skipped: 1 snapshot(s) (2025-10-04T05:00:46Z) - older than the newest snapshot present on sync target
2025-11-28T08:35:59+03:00: re-sync snapshot vm/110/2025-11-01T04:00:52Z
2025-11-28T08:35:59+03:00: no data changes
2025-11-28T08:35:59+03:00: percentage done: 3.94% (5/127 groups)
2025-11-28T08:36:00+03:00: skipped: 1 snapshot(s) (2025-10-04T05:00:03Z) - older than the newest snapshot present on sync target
2025-11-28T08:36:00+03:00: re-sync snapshot vm/111/2025-11-01T04:00:01Z
2025-11-28T08:36:00+03:00: no data changes
2025-11-28T08:36:00+03:00: percentage done: 4.72% (6/127 groups)
2025-11-28T08:36:00+03:00: skipped: 1 snapshot(s) (2025-10-04T05:01:03Z) - older than the newest snapshot present on sync target
2025-11-28T08:36:00+03:00: re-sync snapshot vm/112/2025-11-01T04:01:00Z
2025-11-28T08:36:00+03:00: no data changes
2025-11-28T08:36:00+03:00: percentage done: 5.51% (7/127 groups)
2025-11-28T08:36:00+03:00: re-sync snapshot vm/113/2025-10-04T05:01:47Z
2025-11-28T08:36:00+03:00: sync archive drive-scsi0.img.fidx
2025-11-28T09:38:05+03:00: percentage done: 5.91% (7/127 groups, 1/2 snapshots in group #8)
2025-11-28T09:38:05+03:00: sync group vm/113 failed - failed to upload chunk to s3 backend - upload failed: unexpected status code 503 Service Unavailable
2025-11-28T09:38:05+03:00: skipped: 1 snapshot(s) (2025-10-04T05:07:25Z) - older than the newest snapshot present on sync target
2025-11-28T09:38:06+03:00: re-sync snapshot vm/114/2025-11-01T04:07:33Z
2025-11-28T09:38:06+03:00: no data changes
2025-11-28T09:38:06+03:00: percentage done: 7.09% (9/127 groups)
2025-11-28T09:38:06+03:00: skipped: 1 snapshot(s) (2025-11-16T04:00:56Z) - older than the newest snapshot present on sync target
2025-11-28T09:38:06+03:00: re-sync snapshot vm/115/2025-11-23T04:01:30Z
2025-11-28T09:38:06+03:00: no data changes
 
Last edited:
Code:
root@pbs1:~# journalctl -f | grep SlowDown
Nov 28 09:16:08 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx000008f3463693405d5c8-0069293e28-ec88a7f-default</RequestId><HostId>ec88a7f-default-default</HostId></Error>
Nov 28 09:16:09 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx000000bf934e07c187853-0069293e29-ec88a7f-default</RequestId><HostId>ec88a7f-default-default</HostId></Error>
Nov 28 09:16:52 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx00000a51d33362ba82629-0069293e54-ec4b32a-default</RequestId><HostId>ec4b32a-default-default</HostId></Error>
Nov 28 09:16:53 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx00000c7224a501bf3c7df-0069293e54-ec4b32a-default</RequestId><HostId>ec4b32a-default-default</HostId></Error>
Nov 28 09:16:53 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx00000b55a918144a61b25-0069293e55-ec4b32a-default</RequestId><HostId>ec4b32a-default-default</HostId></Error>
Nov 28 09:16:53 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx000006fecf5961c2c7a68-0069293e55-ec4b32a-default</RequestId><HostId>ec4b32a-default-default</HostId></Error>
Nov 28 09:17:00 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx00000ba3958f92d1e30d4-0069293e5c-ec4b32a-default</RequestId><HostId>ec4b32a-default-default</HostId></Error>
Nov 28 09:18:02 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx00000956a7c19ae4b9252-0069293e9a-ec58f8f-default</RequestId><HostId>ec58f8f-default-default</HostId></Error>
Nov 28 09:19:34 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx00000afee001224d7fcd9-0069293ef6-ec4b32a-default</RequestId><HostId>ec4b32a-default-default</HostId></Error>
Nov 28 09:19:56 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx00000f90c78435f291f9b-0069293f0c-ec58f8f-default</RequestId><HostId>ec58f8f-default-default</HostId></Error>
Nov 28 09:20:22 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx000009ffb00f6300eda65-0069293f26-ec4b32a-default</RequestId><HostId>ec4b32a-default-default</HostId></Error>
Nov 28 09:20:31 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx0000069b7fbdf3513a024-0069293f2f-ec58f8f-default</RequestId><HostId>ec58f8f-default-default</HostId></Error>
Nov 28 09:20:31 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx00000525465e07c88af70-0069293f2f-ec58f8f-default</RequestId><HostId>ec58f8f-default-default</HostId></Error>
Nov 28 09:20:32 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx00000c8b41709abdbe39a-0069293f30-ec58f8f-default</RequestId><HostId>ec58f8f-default-default</HostId></Error>
Nov 28 09:20:32 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx0000053ce118cac8d54ab-0069293f30-ec58f8f-default</RequestId><HostId>ec58f8f-default-default</HostId></Error>
Nov 28 09:21:25 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx00000184de72f63d27ccf-0069293f65-ec88a7f-default</RequestId><HostId>ec88a7f-default-default</HostId></Error>
Nov 28 09:22:34 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx000001747f39b8c7556fa-0069293faa-ec4b32a-default</RequestId><HostId>ec4b32a-default-default</HostId></Error>
Nov 28 09:23:19 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx00000b74d4c9f8a908dd0-0069293fd7-ec58f8f-default</RequestId><HostId>ec58f8f-default-default</HostId></Error>
Nov 28 09:24:02 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx00000768ac14d45084abd-0069294002-ec88a7f-default</RequestId><HostId>ec88a7f-default-default</HostId></Error>
Nov 28 09:24:02 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx000001a0a8dc66d99eb3d-0069294002-ec88a7f-default</RequestId><HostId>ec88a7f-default-default</HostId></Error>
Nov 28 09:25:30 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx00000acd31efcfde52a98-006929405a-ec4b32a-default</RequestId><HostId>ec4b32a-default-default</HostId></Error>
Nov 28 09:25:57 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx0000084c6f592312f05f0-0069294075-ec88a7f-default</RequestId><HostId>ec88a7f-default-default</HostId></Error>
Nov 28 09:28:38 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx00000fc9f95c42fc8ad90-0069294116-ec4b32a-default</RequestId><HostId>ec4b32a-default-default</HostId></Error>
Nov 28 09:30:26 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx0000097015854d6996e01-0069294182-ec58f8f-default</RequestId><HostId>ec58f8f-default-default</HostId></Error>
Nov 28 09:31:22 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx0000026528e9ed2d8ffeb-00692941ba-ec88a7f-default</RequestId><HostId>ec88a7f-default-default</HostId></Error>
Nov 28 09:31:22 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx00000ffc8e5d13db07b90-00692941ba-ec88a7f-default</RequestId><HostId>ec88a7f-default-default</HostId></Error>
Nov 28 09:31:53 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx000008f708f720ff32ba0-00692941d9-ec88a7f-default</RequestId><HostId>ec88a7f-default-default</HostId></Error>
Nov 28 09:32:35 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx0000018590ec79c4eb6ca-0069294203-ec58f8f-default</RequestId><HostId>ec58f8f-default-default</HostId></Error>
Nov 28 09:33:11 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx0000062d9f02118e1fa2d-0069294227-ec88a7f-default</RequestId><HostId>ec88a7f-default-default</HostId></Error>
Nov 28 09:33:55 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx00000b4746a64bc1b367c-0069294253-ec4b32a-default</RequestId><HostId>ec4b32a-default-default</HostId></Error>
Nov 28 09:33:55 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx000005d2672ef19ef2de0-0069294253-ec4b32a-default</RequestId><HostId>ec4b32a-default-default</HostId></Error>
Nov 28 09:33:56 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx0000086b53fa32ff723d5-0069294254-ec4b32a-default</RequestId><HostId>ec4b32a-default-default</HostId></Error>
Nov 28 09:33:56 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx0000001fe7087c722215c-0069294254-ec4b32a-default</RequestId><HostId>ec4b32a-default-default</HostId></Error>
Nov 28 09:34:41 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx000006871f574018d02e8-0069294281-ec58f8f-default</RequestId><HostId>ec58f8f-default-default</HostId></Error>
Nov 28 09:34:41 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx00000d22f947f1ac151cb-0069294281-ec58f8f-default</RequestId><HostId>ec58f8f-default-default</HostId></Error>
Nov 28 09:36:12 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx00000ce90c46aca5eec4d-00692942dc-ec4b32a-default</RequestId><HostId>ec4b32a-default-default</HostId></Error>
Nov 28 09:37:01 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx0000089680882f46fd75a-006929430d-ec88a7f-default</RequestId><HostId>ec88a7f-default-default</HostId></Error>
Nov 28 09:37:01 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx00000e295945a8a2aec36-006929430d-ec88a7f-default</RequestId><HostId>ec88a7f-default-default</HostId></Error>
Nov 28 09:38:01 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx0000036946d753aeb7fbf-0069294349-ec4b32a-default</RequestId><HostId>ec4b32a-default-default</HostId></Error>
Nov 28 09:38:02 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx00000028724b8e2db4b12-006929434a-ec4b32a-default</RequestId><HostId>ec4b32a-default-default</HostId></Error>
Nov 28 09:38:02 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx000000a3e58525d380b81-006929434a-ec4b32a-default</RequestId><HostId>ec4b32a-default-default</HostId></Error>
Nov 28 09:38:02 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx000002866ee8dd9d40a23-006929434a-ec4b32a-default</RequestId><HostId>ec4b32a-default-default</HostId></Error>
Nov 28 09:38:03 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx00000def222ddd9d92358-006929434b-ec4b32a-default</RequestId><HostId>ec4b32a-default-default</HostId></Error>
Nov 28 09:38:03 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx00000721d49d9f943a44f-006929434b-ec4b32a-default</RequestId><HostId>ec4b32a-default-default</HostId></Error>
Nov 28 09:38:03 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx00000ce8fcc63cb2ba546-006929434b-ec4b32a-default</RequestId><HostId>ec4b32a-default-default</HostId></Error>
Nov 28 09:38:04 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx000005b3e1852a666e52e-006929434b-ec4b32a-default</RequestId><HostId>ec4b32a-default-default</HostId></Error>
Nov 28 09:38:04 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx00000b874d0829879d79d-006929434c-ec4b32a-default</RequestId><HostId>ec4b32a-default-default</HostId></Error>
Nov 28 09:55:11 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx000000e05056f1f3baf1d-006929474f-ec88a7f-default</RequestId><HostId>ec88a7f-default-default</HostId></Error>
Nov 28 09:55:11 pbs1 proxmox-backup-proxy[1167]: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SlowDown</Code><BucketName>proxmox-pbs1</BucketName><RequestId>tx000000637528d1b3c4bc8-006929474f-ec88a7f-default</RequestId><HostId>ec88a7f-default-default</HostId></Error>
 
Hi, while there is a retry and backup logic for the PBS s3 client in place, there is still work to be done to also cover there responses correctly. See and subscribe to the tracking issue https://bugzilla.proxmox.com/show_bug.cgi?id=6858

Edit: and just to clarify, if you say with the put-rate-limit in place it is to slow, I would not expect the client to be much faster with the retry logic based on 503 responses in place. It is your Ceph backed S3 API that cannot handle more?
 
Last edited:
Hi, while there is a retry and backup logic for the PBS s3 client in place, there is still work to be done to also cover there responses correctly. See and subscribe to the tracking issue https://bugzilla.proxmox.com/show_bug.cgi?id=6858

Edit: and just to clarify, if you say with the put-rate-limit in place it is to slow, I would not expect the client to be much faster with the retry logic based on 503 responses in place. It is your Ceph backed S3 API that cannot handle more?

When I don't use any put request limit on PBS S3 client side, I can achive 300-500 Mbit/s which is about 10-20 put requests per second. And small VM snapshots (5-10GB) complete without any problem. But its impossible to complete TB size snapshots as PBS S3 client chose to fail instead of back-off a litle and continue to try send as normal.

Edit: " if you say with the put-rate-limit in place it is to slow, I would not expect the client to be much faster with the retry logic based on 503 responses in place" I mean hard limiting S3 client to 1-3 PUT request is slow, as it does not try to send more chunks per second even if remote can handle it most of the times.
 
Last edited: