prune after writing on tape

[Thomas Sossna]

Good day

I'm using PVE for over 7 years and i loved PBS as it came out.
Everything works like a charm, for sure some skills are needed but without them, we would be wrong in our passion anyways.
Thank you in advance for building such unique tools!

Actually, I have a server in a dc in frankfurt running PVE and backing up some local machines there.
The PBS-FRA is synced to the PBS running at my home in Zwickau, let's call its PBS-ZWI.
The sync and everything works fine, PBS-FRA holds the latest 2 backups and PBS-ZWI keeps them longer.
I got an old LTO5 Tapeloader (HP MSL2024) with a bunch of tapes. But I don't have that much HDD space for PBS-ZWI.

My main goal would be to write the PBS-ZWI to the tapes at weekend and then prune the files from PBS-ZWI to free up
space there and archive the tapes.

At the moment im backing up like:
Mo-Sa => rotating daily will be 6 backups per week (will be overwritten on next weeks day)
Sunday => rotating weekly will be 3 packups per month (will be overwritten every month)
fist Sunday month => doing monthly (all of them will be kept. Monthly never gets thrown away)

Is this out of the concept or did I miss something?

Thanks, Servus and take care!

 

[dcsapak]

everytime a new media-set is created (depends on the allocation/retention policy) the datastore will be started to be backed up like it is.
so when you prune backups, all media-sets created afterwards will not contain the pruned backups.

for existing and continued media-sets, the backups already on tape will not be touched, so they stay, regardless what happens on the datastore

so in your scheme, the mo-sa tapes will probably only contain the backups from last week, never longer, the sunday tapes the ones of the last month
and the monthly tapes will contain what was in the datastore at the moment

 

[Thomas Sossna]

everytime a new media-set is created (depends on the allocation/retention policy) the datastore will be started to be backed up like it is.
so when you prune backups, all media-sets created afterwards will not contain the pruned backups.

for existing and continued media-sets, the backups already on tape will not be touched, so they stay, regardless what happens on the datastore

so in your scheme, the mo-sa tapes will probably only contain the backups from last week, never longer, the sunday tapes the ones of the last month
and the monthly tapes will contain what was in the datastore at the moment

Hi Dominik,

thanks for your assistance.
Ok, so as it's only an offsite location from the data center a daily backup is not that needed.
I decided to go on with another solution. Backup2disk the whole week and on Sunday I'll
load some tapes and get rid of the backups on disk.

After writing the whole storage I'll prune the datastore by hand completely, right?
Can this be automated?
So I'll create one continuous set, start the tape job, w8 for the end then I'll prune the datastore...?

And two feature requests, I'll write a post for it if you agree on them
- would there be any way to assign VMs to the desired tape or is this against some best practice rules?
- community plans. So there is in fact your community plan for enterprise users but for me, as an advanced private user, I would be more than happy to give you something back and get access to some more things. But in my case, it would be a dual CPU server for 90€ and then 449€ for my tape drive, that hurts :/ Don't get me wrong, I don't want to discuss the pricing on your software, it's worth every penny for sure!
Alternatively some single payments for tickets? Like know fast track options or something this way?

Thanks and have a great day!

 

[dcsapak]

After writing the whole storage I'll prune the datastore by hand completely, right?
Can this be automated?
So I'll create one continuous set, start the tape job, w8 for the end then I'll prune the datastore...?

you could set the prune job (datastore -> prune &gc) some hours after the tape job, or you write your own script that does first call the tape job and then prune
i'd not completely remove everything though, since the sync/backup will need to upload the chunks again if they were deleted

So I'll create one continuous set, start the tape job, w8 for the end then I'll prune the datastore...?

while possible, note that the chunks are deduplicated on tape too, so having only a single media-set with all backups results in only having one copy of each chunk
on tape. if one tape is corrupted, all backups that need those chunks are not restorable then. it's recommended to have more than one media set of the
desired backups in rotation. if you tune the prune settings, you can just save them staggered, etc. keep 4 weekly/ 12 monthly / 5 yearly etc. it should not take too much space since they are deduplicated anyway

- would there be any way to assign VMs to the desired tape or is this against some best practice rules?

what exactly do you mean by this? how would you "assign" something to a tape?

- community plans. So there is in fact your community plan for enterprise users but for me, as an advanced private user, I would be more than happy to give you something back and get access to some more things. But in my case, it would be a dual CPU server for 90€ and then 449€ for my tape drive, that hurts :/ Don't get me wrong, I don't want to discuss the pricing on your software, it's worth every penny for sure!

just use the version you can/want to afford and give back in ways you can (bugreports, answering uses in the form, etc.) also having only pve and no pbs subscriptions helps too
just note there are no features gated behind the subscriptions, it's all about the (more stable) enterprise repository and ticket-support