AWS tape gateway? or something else?

[rwithd]

I am looking for suggestions.

I have a small number of VM's that are critical for me and I want to offsite my backups to AWS. I would really like to use an amazon Tape Gateway, This makes sense to me because I already have a stack of LTO5 tapes that will need to be migrated. I know there have been some compatibility issues with Tape Gateway.

What I am looking for is; either confirmation that I should put in the time to set up a tape gateway.
-OR-
Suggestions, What other options would you suggest? AWS solutions are preferred but I will take what I can get.

My last resort would be an AWS PBS instance to Sync to maybe... That way I can reduce my data transferred by doing de-duplication locally. But that sounds messy and expensive with licensing.

 

[dcsapak]

we tried to get it to work with the aws tape gateway, and while they are simulating an LTO-5 tape drive, it does not actually emulate all features from that which are necessary for PBS' tape backup.

see
https://forum.proxmox.com/threads/a...tus-failed-failed-to-fill-whole-buffer.93009/
and https://forums.aws.amazon.com/thread.jspa?threadID=343535&tstart=0

if AWS has some kind of fileshare, something like that could work maybe?

My last resort would be an AWS PBS instance to Sync to maybe... That way I can reduce my data transferred by doing de-duplication locally. But that sounds messy and expensive with licensing.

this is actually kind of how pbs was designed. having a second pbs instance off-site which pulls the local snapshots.
regarding licensing, do you mean AWS licensing? no idea on that, but the license on PBS is AGPLv3 (so free & open source), and the subscriptions are for support and the enterprise repo only

 

[hvisage]

this is actually kind of how pbs was designed. having a second pbs instance off-site which pulls the local snapshots.

Using PBS in exactly this manner!
actually 2 ways in my case
1 the local backup PBS (having the historical long term backups too)
2a DR DC (same provider) syncing from the local shorter term pruning
2b other side of the world on a different provider syncing also from the local, even more pruning as it's a total disaster restore case.

 

[rwithd]

this is actually kind of how pbs was designed. having a second pbs instance off-site which pulls the local snapshots.
regarding licensing, do you mean AWS licensing? no idea on that, but the license on PBS is AGPLv3 (so free & open source), and the subscriptions are for support and the enterprise repo only

Ok, That makes sense,

So lets say I have a server that backs up every hour, could I limit the transfer to only the last backup of the day? or would I be forced to back up all of the individual, albeit smaller backups?

 

[dcsapak]

could I limit the transfer to only the last backup of the day? or would I be forced to back up all of the individual, albeit smaller backups?

currently not, but the backups are deduplicated, so the sync only pulls unique chunks (i.e. if there are 2 backups with 2GiB data each, but they share 1GiB chunks, you'd have to pull 3GiB)

but that would probably a good feature request: https://bugzilla.proxmox.com

 

[rwithd]

Thanks, Dominik. I am going to simulate this setup and explore before submitting a feature request.

 

[hvisage]

Well, for the deduplication to work, you need at least a backup as close to the current as possible, but the way I "simulate" the above, is to do frequent synchronizations, but then do a pruning of all the old ones I don't want on the remote. Also PBS will NOT sync previous old backups (ie, your remote have 2022/02/28, but the local have 2022/03/01-02 and 2022/02/20 to 2022/02/28, the remote will ignore (and you'll see the information messages in the logs) that it will ignore all those older then 202202/28 (the newest on remote) and only synchronize 2022/03/01 & 02