Hi,
Just read this in the documentation: https://pbs.proxmox.com/docs/managing-remotes.html
Is that technically not possible or just not wanted?
Because I'm thinking how to make my PBS redundant and this would be a problem.
Right now I'm doing once per day from Monday to Saturday a daily "snapshot mode" backup of all guests to my datastore1 with a retention of 3 last + 7 daily.
Sundays I run a weekly "stop mode" backup of all guests to my datastore2 with a retention of 3 last, 4 weekly, 12 monthly, 3 yearly.
And every time before changing any configurations of a guest I manually create a "stop mode" backup of that guest, set the protected flag so it will be skipped by pruning and never be deleted, until I manually delete it months later if not longer needed, and I will explain in the backups notes what software versions that guest is running and what I am planning to change. These manual protected backups I also store on my datastore2 for longterm backups.
Its so great that PBS is based on deduplication, unlike most other solutions, so it doesn't matter that much how many backups of the same guest you want to keep. But I always run into situations where I want to achieve something, that can only be done by adding a new datastore so deduplication isn't that great anymore because I then still need to store the same chunks multiple times, because the same guest is stored in all datstores.
I only got two datastores because the pruning is not flexible enough. I want my long term backups (monthly/yearly) to be the weekly "stop mode" backups only and not the less safe daily "snapshot mode" backups but there is no way to achieve this without creating a additional datastore just for daily "snapshot mode" backups. Because otherwise it is just gambling if the last day of the month or last day of the year is a sunday or not. So statistically only a few of the monthly/yearly backups would be the "stop mode" ones.
And now it looks like I would need to add a third datastore just for storing manual backups, so every chunk needs to be stored 3 times.
My idea was to setup a second PBS that pulls once per week all backup snapshots of datastore2 (the one for manual/weekly/monthly/yearly backups) from the main PBS with disabled "remove-vanished" and a local datastore retention matching the one on the main PBS. That way I would get a additional protection because even if someone gets access to the main PBS he couldn't delete backups from the second PBS without getting access to that too.
But then I would have the problem that the GC on the second PBS would delete my manual backups, because they wouldn't longer be ignored by the pruning task, when the protected flag get lost while syncing.
So the only option I see is to add a third datastore with "keep-all" as retention and save all my manual backups, that should never be automatically pruned, there instead. Because then the protected flag isn't needed any longer.
I really hope the new tagging features, its worked on, will allow to be more versatile when it comes to retention. Like automatically adding tags to snapshots by the corresponding backup job on the PVE and a ability to add tags to manual backups. And then telling the prune task just to prune snapshots with (or without) specific tags.
If I could tag backups as "daily" (by my daily "snapshot mode" backup task), "weekly" (by my weekly "stop mode" backup task) and "manual" for my manual backups and tell the pruning to just include or exclude these tags, then I could use the same single datastore for everything and wouldn't waste 66% of the backup storages capacity.
Then something like this would be great and everything could be done with a single datastore:
Edit:
Also found this:
https://pbs.proxmox.com/docs/backup-client.html#backup-pruning:
So as an alternative I could write my own script using the API to compare all backup snapshots on both PBS and set the protected flag again if it is missing on the pulling PBS. So looks like it would be possible to also sync the protected flag. So whats the reason that this flag won't be synced?
Just read this in the documentation: https://pbs.proxmox.com/docs/managing-remotes.html
Is that technically not possible or just not wanted?
Because I'm thinking how to make my PBS redundant and this would be a problem.
Right now I'm doing once per day from Monday to Saturday a daily "snapshot mode" backup of all guests to my datastore1 with a retention of 3 last + 7 daily.
Sundays I run a weekly "stop mode" backup of all guests to my datastore2 with a retention of 3 last, 4 weekly, 12 monthly, 3 yearly.
And every time before changing any configurations of a guest I manually create a "stop mode" backup of that guest, set the protected flag so it will be skipped by pruning and never be deleted, until I manually delete it months later if not longer needed, and I will explain in the backups notes what software versions that guest is running and what I am planning to change. These manual protected backups I also store on my datastore2 for longterm backups.
Its so great that PBS is based on deduplication, unlike most other solutions, so it doesn't matter that much how many backups of the same guest you want to keep. But I always run into situations where I want to achieve something, that can only be done by adding a new datastore so deduplication isn't that great anymore because I then still need to store the same chunks multiple times, because the same guest is stored in all datstores.
I only got two datastores because the pruning is not flexible enough. I want my long term backups (monthly/yearly) to be the weekly "stop mode" backups only and not the less safe daily "snapshot mode" backups but there is no way to achieve this without creating a additional datastore just for daily "snapshot mode" backups. Because otherwise it is just gambling if the last day of the month or last day of the year is a sunday or not. So statistically only a few of the monthly/yearly backups would be the "stop mode" ones.
And now it looks like I would need to add a third datastore just for storing manual backups, so every chunk needs to be stored 3 times.
My idea was to setup a second PBS that pulls once per week all backup snapshots of datastore2 (the one for manual/weekly/monthly/yearly backups) from the main PBS with disabled "remove-vanished" and a local datastore retention matching the one on the main PBS. That way I would get a additional protection because even if someone gets access to the main PBS he couldn't delete backups from the second PBS without getting access to that too.But then I would have the problem that the GC on the second PBS would delete my manual backups, because they wouldn't longer be ignored by the pruning task, when the protected flag get lost while syncing.
So the only option I see is to add a third datastore with "keep-all" as retention and save all my manual backups, that should never be automatically pruned, there instead. Because then the protected flag isn't needed any longer.
I really hope the new tagging features, its worked on, will allow to be more versatile when it comes to retention. Like automatically adding tags to snapshots by the corresponding backup job on the PVE and a ability to add tags to manual backups. And then telling the prune task just to prune snapshots with (or without) specific tags.
If I could tag backups as "daily" (by my daily "snapshot mode" backup task), "weekly" (by my weekly "stop mode" backup task) and "manual" for my manual backups and tell the pruning to just include or exclude these tags, then I could use the same single datastore for everything and wouldn't waste 66% of the backup storages capacity.
Then something like this would be great and everything could be done with a single datastore:
Code:
proxmox-backup-client prune MySingleDatastore --keep-last 3 --keep-daily 7 --include-tag "daily" --exclude-tag "manual"
proxmox-backup-client prune MySingleDatastore --keep-last 3 --keep-weekly 4 --keep-monthly 12 --keep-yearly 3 --include-tag "weekly" --exclude-tag "manual"Edit:
Also found this:
https://pbs.proxmox.com/docs/backup-client.html#backup-pruning:
So as an alternative I could write my own script using the API to compare all backup snapshots on both PBS and set the protected flag again if it is missing on the pulling PBS. So looks like it would be possible to also sync the protected flag. So whats the reason that this flag won't be synced?
Last edited:
