Anyone have a working solution to backup to S3?

[tyaz]

I've been trying to get PBS to use a S3 bucket for the past month or so. Most of the time, it works fine but randomly different .chunks dirs go missing and then backups get corrupted.

I assume this isn't an issue with proxmox itself, but moreso with rclone and not being able to write/read the S3 bucket.

Has anyone been successful in using S3 with PBS? I use S3 for full backups outside of PBS because it's one file and it works great. But, since PBS uses many different small chunk files, it seems it's not super reliable.

Perhaps there's a better solution than using rclone and mounting /backup to the S3 bucket?

 

[Johannes S]

Unfortunely it's not possible at the moment, rclone corrupts backups as you noticed, wrappers like s3fs are unsupported hacks I wouldn't trust anything important with:
https://forum.proxmox.com/threads/using-an-amazon-aws-s3-bucket-as-backup-storage.133555/post-748718

Your best bet is using a PBS as a service provider like Inett or tuxis.nl or renting a vserver or dedicated server ( depending on the needed size) and running PBS on it. Then you can use PBS remote sync feature.
If you absolutely want to use rclone stick with normal vzdump backups without PBS, they will work but need more space

 

[tyaz]

Unfortunely it's not possible at the moment, rclone corrupts backups as you noticed, wrappers like s3fs are unsupported hacks I wouldn't trust anything important with:
https://forum.proxmox.com/threads/using-an-amazon-aws-s3-bucket-as-backup-storage.133555/post-748718

Your jest bet is using a PBS as a service provider like Inett or tuxil.nl or renting a vserver or dedicated server ( depending on the needed size) and running PBS on it. Then you can use PBS remote sync feature.
If yoz absolutely want zo use rclone Stick with normal vzdump backups without PBS, they will work but need more space

Thanks... Unfortunate they haven't added S3 support into PBS... S3 seems like such a better way of doing backups these days.

Do you know of any PBS as a Service providers in the US? https://remote-backups.com/ looks nice, but it's in Europe... All of our servers are in the US, so sending all that data to Europe isn't ideal.

 

[Johannes S]

Thanks... Unfortunate they haven't added S3 support into PBS... S3 seems like such a better way of doing backups these days.

It's on the Roadmap for years but as far I know they havn't started work on it yet.

Do you know of any PBS as a Service providers in the US? https://remote-backups.com/ looks nice, but it's in Europe... All of our servers are in the US, so sending all that data to Europe isn't ideal.

Then I would use a vserver provider in the US. Amother option might be to use external storage Media as removable datastore and store them offsite.

If you have multiple locations of your servers you could also setup one PBS per location and sync between them. This would be my preffered setup.