Problem with restoring backups from s3 endpoint

N

nilforoosh

New Member
Apr 17, 2024
2
1
3
Hello. We recently upgraded PBS to the new version and we want to use the S3 feature for storing backups. The issue is that backups are taken correctly and even verified, but during restore we get a connection error. A sample log can be seen below.


Using encryption key from file descriptor..
Fingerprint: 80:52:e3:24:a4:62:8a:91
Using encryption key from file descriptor..
Fingerprint: 80:52:e3:24:a4:62:8a:91
Using encryption key from file descriptor..
Fingerprint: 80:52:e3:24:a4:62:8a:91
new volume ID is 'ceph:vm-119-disk-0'
restore proxmox backup image: /usr/bin/pbs-restore --repository backup@pbs@192.168.1.10:arvancloud vm/120/2025-09-09T17:25:42Z drive-scsi0.img.fidx 'rbd:ceph/vm-119-disk-0:conf=/etc/pve/ceph.conf:id=admin:keyring=/etc/pve/priv/ceph/ceph.keyring' --verbose --format raw --keyfile /etc/pve/priv/storage/arvancloud.enc --skip-zero
connecting to repository 'backup@pbs@192.168.1.10:arvancloud'
using up to 4 threads
open block backend for target 'rbd:ceph/vm-119-disk-0:conf=/etc/pve/ceph.conf:id=admin:keyring=/etc/pve/priv/ceph/ceph.keyring'
starting to restore snapshot 'vm/120/2025-09-09T17:25:42Z'
download and verify backup index
fetching up to 16 chunks in parallel
progress 1% (read 2147483648 bytes, zeroes = 83% (1803550720 bytes), duration 27 sec)
restore failed: client error (Connect)
Removing image: 1% complete...
Removing image: 2% complete...
Removing image: 3% complete...
Removing image: 4% complete...
Removing image: 5% complete...
Removing image: 6% complete...
Removing image: 7% complete...
Removing image: 8% complete...
Removing image: 9% complete...
Removing image: 10% complete...
Removing image: 11% complete...
Removing image: 12% complete...
Removing image: 13% complete...
Removing image: 14% complete...
Removing image: 15% complete...
Removing image: 16% complete...
Removing image: 17% complete...
Removing image: 18% complete...
Removing image: 19% complete...
Removing image: 20% complete...
Removing image: 21% complete...
Removing image: 22% complete...
Removing image: 23% complete...
Removing image: 24% complete...
Removing image: 25% complete...
Removing image: 26% complete...
Removing image: 27% complete...
Removing image: 28% complete...
Removing image: 29% complete...
Removing image: 30% complete...
Removing image: 31% complete...
Removing image: 32% complete...
Removing image: 33% complete...
Removing image: 34% complete...
Removing image: 35% complete...
Removing image: 36% complete...
Removing image: 37% complete...
Removing image: 38% complete...
Removing image: 39% complete...
Removing image: 40% complete...
Removing image: 41% complete...
Removing image: 42% complete...
Removing image: 43% complete...
Removing image: 44% complete...
Removing image: 45% complete...
Removing image: 46% complete...
Removing image: 47% complete...
Removing image: 48% complete...
Removing image: 49% complete...
Removing image: 50% complete...
Removing image: 51% complete...
Removing image: 52% complete...
Removing image: 53% complete...
Removing image: 54% complete...
Removing image: 55% complete...
Removing image: 56% complete...
Removing image: 57% complete...
Removing image: 58% complete...
Removing image: 59% complete...
Removing image: 60% complete...
Removing image: 61% complete...
Removing image: 62% complete...
Removing image: 63% complete...
Removing image: 64% complete...
Removing image: 65% complete...
Removing image: 66% complete...
Removing image: 67% complete...
Removing image: 68% complete...
Removing image: 69% complete...
Removing image: 70% complete...
Removing image: 71% complete...
Removing image: 72% complete...
Removing image: 73% complete...
Removing image: 74% complete...
Removing image: 75% complete...
Removing image: 76% complete...
Removing image: 77% complete...
Removing image: 78% complete...
Removing image: 79% complete...
Removing image: 80% complete...
Removing image: 81% complete...
Removing image: 82% complete...
Removing image: 83% complete...
Removing image: 84% complete...
Removing image: 85% complete...
Removing image: 86% complete...
Removing image: 87% complete...
Removing image: 88% complete...
Removing image: 89% complete...
Removing image: 90% complete...
Removing image: 91% complete...
Removing image: 92% complete...
Removing image: 93% complete...
Removing image: 94% complete...
Removing image: 95% complete...
Removing image: 96% complete...
Removing image: 97% complete...
Removing image: 98% complete...
Removing image: 99% complete...
Removing image: 100% complete...done.
temporary volume 'ceph:vm-119-disk-0' successfully removed
error before or during data restore, some or all disks were not completely restored. VM 119 state is NOT cleaned up.
TASK ERROR: command '/usr/bin/pbs-restore --repository backup@pbs@192.168.1.10:arvancloud vm/120/2025-09-09T17:25:42Z drive-scsi0.img.fidx 'rbd:ceph/vm-119-disk-0:conf=/etc/pve/ceph.conf:id=admin:keyring=/etc/pve/priv/ceph/ceph.keyring' --verbose --format raw --keyfile /etc/pve/priv/storage/arvancloud.enc --skip-zero' failed: exit code 255


It seems like a network issue. But what’s strange is that taking backups works without any problem. Verification also works fine. But during restore, it fails.

The log related to the corresponding task on the backup server is as follows.

2025-09-14T16:47:59+03:30: starting new backup reader datastore 'arvancloud': "/mnt/datastore/backup/arvancloud"
2025-09-14T16:47:59+03:30: protocol upgrade done
2025-09-14T16:47:59+03:30: GET /download
2025-09-14T16:47:59+03:30: download "/mnt/datastore/backup/arvancloud/vm/109/2025-09-10T09:03:16Z/index.json.blob"
2025-09-14T16:47:59+03:30: GET /download
2025-09-14T16:47:59+03:30: download "/mnt/datastore/backup/arvancloud/vm/109/2025-09-10T09:03:16Z/drive-scsi0.img.fidx"
2025-09-14T16:47:59+03:30: register chunks in 'drive-scsi0.img.fidx' as downloadable.
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:47:59+03:30: GET /chunk
2025-09-14T16:48:00+03:30: GET /chunk
2025-09-14T16:48:00+03:30: GET /chunk
2025-09-14T16:48:00+03:30: GET /chunk
2025-09-14T16:48:00+03:30: GET /chunk
2025-09-14T16:48:00+03:30: GET /chunk
2025-09-14T16:48:00+03:30: GET /chunk
2025-09-14T16:48:00+03:30: GET /chunk
2025-09-14T16:48:00+03:30: GET /chunk
2025-09-14T16:48:00+03:30: GET /chunk
2025-09-14T16:48:00+03:30: GET /chunk
2025-09-14T16:48:00+03:30: GET /chunk
2025-09-14T16:48:00+03:30: GET /chunk
2025-09-14T16:48:00+03:30: GET /chunk
2025-09-14T16:48:00+03:30: GET /chunk
2025-09-14T16:48:00+03:30: GET /chunk
2025-09-14T16:48:01+03:30: GET /chunk
2025-09-14T16:48:01+03:30: GET /chunk
2025-09-14T16:48:01+03:30: GET /chunk
2025-09-14T16:48:01+03:30: GET /chunk
2025-09-14T16:48:01+03:30: GET /chunk
2025-09-14T16:48:01+03:30: GET /chunk
2025-09-14T16:48:01+03:30: GET /chunk
2025-09-14T16:48:01+03:30: GET /chunk
2025-09-14T16:48:01+03:30: GET /chunk
2025-09-14T16:48:01+03:30: GET /chunk
2025-09-14T16:48:01+03:30: GET /chunk
2025-09-14T16:48:01+03:30: GET /chunk
2025-09-14T16:48:01+03:30: GET /chunk
2025-09-14T16:48:01+03:30: GET /chunk
2025-09-14T16:48:01+03:30: GET /chunk
2025-09-14T16:48:01+03:30: GET /chunk
2025-09-14T16:48:01+03:30: GET /chunk
2025-09-14T16:48:01+03:30: GET /chunk
2025-09-14T16:48:01+03:30: GET /chunk
2025-09-14T16:48:01+03:30: GET /chunk
2025-09-14T16:48:01+03:30: GET /chunk
2025-09-14T16:48:01+03:30: GET /chunk
2025-09-14T16:48:01+03:30: GET /chunk
2025-09-14T16:48:01+03:30: GET /chunk
2025-09-14T16:48:01+03:30: GET /chunk
2025-09-14T16:48:02+03:30: GET /chunk
2025-09-14T16:48:02+03:30: GET /chunk
2025-09-14T16:48:02+03:30: GET /chunk
2025-09-14T16:48:02+03:30: GET /chunk
2025-09-14T16:48:02+03:30: GET /chunk: 400 Bad Request: request timeout
2025-09-14T16:48:02+03:30: GET /chunk
2025-09-14T16:48:02+03:30: GET /chunk
2025-09-14T16:48:02+03:30: GET /chunk
2025-09-14T16:48:02+03:30: GET /chunk
2025-09-14T16:48:02+03:30: GET /chunk
2025-09-14T16:48:02+03:30: GET /chunk
2025-09-14T16:48:02+03:30: GET /chunk
2025-09-14T16:48:02+03:30: GET /chunk
2025-09-14T16:48:02+03:30: TASK ERROR: connection error: connection reset
 
Last edited:
Hi everyone,
I’m experiencing the exact same issue using the latest versions of Proxmox VE and Proxmox Backup Server (PBS). I’ve configured an S3-compatible storage for backups.
Backups are created successfully and even the verification process completes without any errors. However, when I try to restore a backup, I get the error:
restore failed: client error (Connect)
The logs are very similar to what was shared above. I’ve double-checked all settings, including the S3 configuration, encryption key file, and network access — everything looks correct, but the restore still fails.
If anyone has encountered this issue or found a workaround, I’d really appreciate your help.
 
Last edited:
  • Like
Reactions: nilforoosh
Last edited:
You must log in or register to reply here.
Top Bottom