AWS tape gateway? or something else?

R

rwithd

Member
Proxmox Subscriber
Dec 8, 2021
22
0
6
32
USA
I am looking for suggestions.

I have a small number of VM's that are critical for me and I want to offsite my backups to AWS. I would really like to use an amazon Tape Gateway, This makes sense to me because I already have a stack of LTO5 tapes that will need to be migrated. I know there have been some compatibility issues with Tape Gateway.

What I am looking for is; either confirmation that I should put in the time to set up a tape gateway.
-OR-
Suggestions, What other options would you suggest? AWS solutions are preferred but I will take what I can get.

My last resort would be an AWS PBS instance to Sync to maybe... That way I can reduce my data transferred by doing de-duplication locally. But that sounds messy and expensive with licensing.
 
we tried to get it to work with the aws tape gateway, and while they are simulating an LTO-5 tape drive, it does not actually emulate all features from that which are necessary for PBS' tape backup.

see
https://forum.proxmox.com/threads/a...tus-failed-failed-to-fill-whole-buffer.93009/
and https://forums.aws.amazon.com/thread.jspa?threadID=343535&tstart=0

if AWS has some kind of fileshare, something like that could work maybe?

rwithd said:
My last resort would be an AWS PBS instance to Sync to maybe... That way I can reduce my data transferred by doing de-duplication locally. But that sounds messy and expensive with licensing.
Click to expand...
this is actually kind of how pbs was designed. having a second pbs instance off-site which pulls the local snapshots.
regarding licensing, do you mean AWS licensing? no idea on that, but the license on PBS is AGPLv3 (so free & open source), and the subscriptions are for support and the enterprise repo only
 
  • Like
Reactions: hvisage
dcsapak said:
this is actually kind of how pbs was designed. having a second pbs instance off-site which pulls the local snapshots.
Click to expand...

Using PBS in exactly this manner!
actually 2 ways in my case
1 the local backup PBS (having the historical long term backups too)
2a DR DC (same provider) syncing from the local shorter term pruning
2b other side of the world on a different provider syncing also from the local, even more pruning as it's a total disaster restore case.
 
dcsapak said:
this is actually kind of how pbs was designed. having a second pbs instance off-site which pulls the local snapshots.
regarding licensing, do you mean AWS licensing? no idea on that, but the license on PBS is AGPLv3 (so free & open source), and the subscriptions are for support and the enterprise repo only
Click to expand...

Ok, That makes sense,

So lets say I have a server that backs up every hour, could I limit the transfer to only the last backup of the day? or would I be forced to back up all of the individual, albeit smaller backups?
 
rwithd said:
could I limit the transfer to only the last backup of the day? or would I be forced to back up all of the individual, albeit smaller backups?
Click to expand...
currently not, but the backups are deduplicated, so the sync only pulls unique chunks (i.e. if there are 2 backups with 2GiB data each, but they share 1GiB chunks, you'd have to pull 3GiB)

but that would probably a good feature request: https://bugzilla.proxmox.com
 
Well, for the deduplication to work, you need at least a backup as close to the current as possible, but the way I "simulate" the above, is to do frequent synchronizations, but then do a pruning of all the old ones I don't want on the remote. Also PBS will NOT sync previous old backups (ie, your remote have 2022/02/28, but the local have 2022/03/01-02 and 2022/02/20 to 2022/02/28, the remote will ignore (and you'll see the information messages in the logs) that it will ignore all those older then 202202/28 (the newest on remote) and only synchronize 2022/03/01 & 02
 
You must log in or register to reply here.
Top Bottom