PBS tape backup - various questions

[flames]

Hello dear Proxmox-Team,

i am still playing around with tape backup, currently with LTO-4, but even if i would invest considerable money into a LTO-8 (i would better put the money into a proxmox subscription), i would still hit the same questions, since the data amount is growing.

- a quote from PBS documentation https://pbs.proxmox.com/docs/tape-backup.html "Drive performance ... The only way to speed that data rate up is to use more than one drive. That way, you can run several backup jobs in parallel, or run restore jobs while the other dives are used for backups."
-- question 1 - since a tape job in PBS can only backup a datastore as a whole, i would need multiple datastores to split the job to multiple drives (and sacrifice deduplication)? or is it possible to run a single datastore-tape-job to multiple drives at once?
-- question 2 - the roadmap feature about namespaces will allow to backup multiple pve hosts/clusters to a single datastore, so there is no conflict with IDs + shared chunks for deduplication over those namespaces, but will it allow to have a tape job per namespace then or still per datastore?
-- question 3 - i searched through tickets, but probably used wrong keywords. will it be possible to specify a snapshot range for tape jobs? to explain, a tape job runs multiple days (its ok, daily pve to pbs, weekly pbs to tape). garbage collection is covered by 24h+5min last access before removes chunks. but since pve to pbs jobs are running daily (and are very fast), new daily snapshots are added faster than tape job can write them away (when a new snapshot is ready it will be written to tapes of not yet taped groups. got in a loop there. also with lto-8 i would get into the same loop even at full theoretical speed of 360MBs)
-- question 4 - addition to q3 - pbs is responsible for daily retention (in my case 30d/52w/3y) - so no need to have the whole datastore on weekly tapes. cool feature would be to limit the range or a specific day of snapshot in the tape job, so it would write only that to tape, even if datastore receives new snapshots meanwhile.

thanks in advance!

 

[dietmar]

-- question 1 - since a tape job in PBS can only backup a datastore as a whole, i would need multiple datastores to split the job to multiple drives (and sacrifice deduplication)? or is it possible to run a single datastore-tape-job to multiple drives at once?

Yes, you currently need multiple datastores.

-- question 2 - the roadmap feature about namespaces will allow to backup multiple pve hosts/clusters to a single datastore, so there is no conflict with IDs + shared chunks for deduplication over those namespaces, but will it allow to have a tape job per namespace then or still per datastore?

We plan to add filters for tape backup jobs, so that you can backup parts of a datastore...

-- question 3 - i searched through tickets, but probably used wrong keywords. will it be possible to specify a snapshot range for tape jobs?

see above (we plan to add filters like that)

-- question 4 - addition to q3 - pbs is responsible for daily retention (in my case 30d/52w/3y) - so no need to have the whole datastore on weekly tapes. cool feature would be to limit the range or a specific day of snapshot in the tape job, so it would write only that to tape, even if datastore receives new snapshots meanwhile.

Our tape media-set always contains the whole datastore (but fully deduplicated). This is different from traditional approaches with incremental tape backups, but makes restore much simpler because you only need one media-set.