PBS S3 backend behind Cloudflare Tunnel fails with SignatureDoesNotMatch against MinIO, while direct local access works

[adadbrd]

Component
s3 / datastore

Version
4.1.4

Severity
normal

Summary
When configuring a PBS S3 backend against a self-hosted MinIO (or Garage) instance exposed through Cloudflare Tunnel, PBS fails with SignatureDoesNotMatch.
The same credentials work locally on the MinIO host, and a Synology NAS can also access the same MinIO endpoint successfully.
This suggests the issue is related to the S3 request/signature path through Cloudflare Tunnel, and PBS currently does not expose a compatibility option similar to rclone’s --s3-sign-accept-encoding=false.

Environment

• Proxmox Backup Server installed in a VM
• MinIO self-hosted in Docker
• MinIO exposed externally only through Cloudflare Tunnel
• No direct LAN access from PBS to the MinIO host
• S3 endpoint configured in PBS with:
  • HTTPS
  • public hostname through Cloudflare Tunnel
  • path-style enabled
  • region us-east-1
• MinIO has no explicit MINIO_SITE_REGION configured
• Buckets exist and are accessible locally on the MinIO host

MinIO side validation
On the MinIO host, using MinIO client locally with the same credentials works correctly:


mc ls local
mc ls local/pbs

This lists the buckets correctly, including pbs.

Also, MINIO_SITE_REGION is not set:



docker exec -it minio env | grep -i MINIO_SITE_REGION


No output is returned, so MinIO appears to use its default region behavior.

PBS configuration used
S3 endpoint configuration in PBS:

• Endpoint: miniobackups3.xxx.fr
• Port: 443
• Region: us-east-1
• Path Style: enabled
• Credentials: valid and tested separately
• Bucket: pbs

Observed PBS errors
PBS fails when listing buckets and when attempting to create the datastore:



<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><Resource>/</Resource><RequestId>...</RequestId><HostId>...</HostId></Error>

GET /api2/json/config/s3/BackupNuc/list-buckets: 400 Bad Request: failed to list buckets


And via CLI:



proxmox-backup-manager datastore create pbs-s3 /var/lib/proxmox-backup/cache-pbs-s3 \
--backend type=s3,client=BackupNuc,bucket=pbs


Result:



Error: failed to access bucket: bucket does not exist or no permission to access it


Server-side logs from MinIO / S3 endpoint
The S3 backend receives the request, but rejects it as an invalid signature:



GET / -> SignatureDoesNotMatch
Forbidden: Invalid signature


Independent reproduction outside PBS
The same problem can be reproduced outside PBS using rclone from a remote machine through the exact same Cloudflare hostname:



rclone lsd :s3bs \
--s3-provider Other \
--s3-env-auth=false \
--s3-access-key-id 'REDACTED' \
--s3-secret-access-key 'REDACTED' \
--s3-region 'us-east-1' \
--s3-endpoint 'https://miniobackups3.xxx.fr' \
--s3-force-path-style=true \
-vv


Result:



ERROR : error listing: operation error S3: ListObjects, https response error StatusCode: 403, api error SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided.


This shows the problem is reproducible independently of PBS when using this proxied endpoint.

Important comparison

• Local direct access to MinIO on the MinIO host works
• Synology NAS access to the same MinIO endpoint works
• PBS fails
• rclone fails through Cloudflare with the same signature error

This suggests that some S3 clients tolerate or avoid the proxy-altered request path, while PBS currently does not.

Expected behavior
PBS should either:

1. work correctly with S3-compatible endpoints behind Cloudflare Tunnel / similar proxies, or
2. expose a compatibility option for problematic signed headers, similar in spirit to rclone’s --s3-sign-accept-encoding=false, or
3. explicitly document this setup as unsupported if it cannot be supported reliably

Actual behavior
PBS S3 backend fails with SignatureDoesNotMatch when accessing MinIO through Cloudflare Tunnel, even though:

• credentials are valid
• bucket exists
• local direct access works

Why this looks like a PBS limitation rather than only a configuration issue
After validation:

• credentials are confirmed valid locally
• region appears correct (us-east-1)
• path-style is enabled
• bucket exists
• failure still occurs through the proxied public endpoint

PBS currently does not appear to provide a client compatibility setting for this class of proxy/signature issue.

Request
Please investigate whether PBS can support this proxy/tunnel scenario, or add an S3 client compatibility option for altered signed headers in proxied environments.
If this setup is intentionally unsupported, it would help to document that explicitly.

Additional notes
The issue was first seen with bucket listing in the UI, but it is not limited to UI listing. It is reproducible with direct bucket operations as well.

Sanitized reproduction steps

1. Deploy MinIO behind Cloudflare Tunnel with HTTPS hostname
2. Create a bucket pbs
3. Verify local access on the MinIO host using mc ls local/pbs
4. Configure a PBS S3 endpoint against the public Cloudflare hostname
5. Set region us-east-1
6. Enable path-style
7. Try listing buckets or creating a datastore
8. Observe SignatureDoesNotMatch