Different backup retentions

[Dunuin]

Hi,

So how are the new backup retentions working?

Until now the documentation recommended to set retentions PBS for the datastore and not using retentions through the PVE WebUI Backup tasks.
I updated my PBS from 2.0.13 to 2.0.14 but I can't see that anything changed except for that new "protect backup" button?
So If I want multiple backups jobs with different retentions using the same PBS datastore I need to remove the retentions in PBS for the datastore and only set the retentions in the PVE backup job configuration?
And what about the proxmox-backup-client? I guess there are now additional arguments to set the "do not prune" flag and to set individual retentions?

 

[fabian]

Hi,

So how are the new backup retentions working?

Until now the documentation recommended to set retentions PBS for the datastore and not using retentions through the PVE WebUI Backup tasks.
I updated my PBS from 2.0.13 to 2.0.14 but I can't see that anything changed except for that new "protect backup" button?

well on the PBS side, only protection is new

So If I want multiple backups jobs with different retentions using the same PBS datastore I need to remove the retentions in PBS for the datastore and only set the retentions in the PVE backup job configuration?

yes

And what about the proxmox-backup-client? I guess there are now additional arguments to set the "do not prune" flag and to set individual retentions?

yes, you can set/clear the protected flag (proxmox-backup-client snapshot protected update ..), and you can run a manual prune after a backup (proxmox-backup-client prune ..., that has always been possible, it's not a new feature vzdump will now call this if job or storage specific prune settings are set, after the backup itself)

 

[Dunuin]

I think I hit a bug? I tried to edit mit old PVE Backup Job and hit the Ok button and the backup job disappeared from the list.
Can I somehow verify that the backup job is really removed and not just invisible in the WebUI?

 

[fabian]

I think I hit a bug? I tried to edit mit old PVE Backup Job and hit the Ok button and the backup job disappeared from the list.
Can I somehow verify that the backup job is really removed and not just invisible in the WebUI?

previously the vzdump jobs were stored in /etc/pve/vzdump.cron , with PVE 7.1 and the new scheduler they are switched to /etc/pve/jobs.cfg

 

[Dunuin]

And then I wanted to create a new backup job, because the old one disappeared after editing it, but I get this "Use of inherited AUTOLOAD for non-method PVE::API2::Backup::uuid() is no longer allowed at /usr/share/perl5/PVE/API2/Backup.pm line 198. (500)".

Edit:

cat /etc/pve/vzdump.cron
# cluster wide vzdump cron schedule
# Automatically generated file - do not edit

PATH="/usr/sbin:/usr/bin:/sbin:/bin"

cat /etc/pve/jobs.cfg
vzdump: a0364706d6d48a82f23127bbb4e994120153cb95:2
        schedule satmontuewedthufri 05:30
        all 1
        enabled 1
        exclude 123,125,103
        mailnotification failure
        mailto my@mail.de
        mode snapshot
        quiet 1
        storage PBS

So there is a backup job but the WebUI isn'T showing anything:

 

[fabian]

And then I wanted to create a new backup job, because the old one disappeared after editing it, but I get this "Use of inherited AUTOLOAD for non-method PVE::API2::Backup::uuid() is no longer allowed at /usr/share/perl5/PVE/API2/Backup.pm line 198. (500)".

Edit:

cat /etc/pve/vzdump.cron
# cluster wide vzdump cron schedule
# Automatically generated file - do not edit

PATH="/usr/sbin:/usr/bin:/sbin:/bin"

cat /etc/pve/jobs.cfg
vzdump: a0364706d6d48a82f23127bbb4e994120153cb95:2
        schedule satmontuewedthufri 05:30
        all 1
        enabled 1
        exclude 123,125,103
        mailnotification failure
        mailto my@mail.de
        mode snapshot
        quiet 1
        storage PBS

So there is a backup job but the WebUI isn'T showing anything.

https://lists.proxmox.com/pipermail/pve-devel/2021-November/051024.html

 

[fabian]

are all your nodes properly updated?

 

[Dunuin]

Yes, did a successful apt update && apt full-upgrade && reboot for both PBS and PVE just some minutes ago. And a STRG+F5 in the browser.

I look if I find something in the syslog.

 

[Dunuin]

There are alot of lines like this:

Nov 17 13:52:00 Hypervisor pvescheduler[80558]: file /etc/pve/jobs.cfg line 2 (section 'a0364706d6d48a82f23127bbb4e994120153cb95:2') - unable to parse value of 'schedule': invalid format - invalid calendar event 'sat#000mon#000tue#000wed#000thu#000fri 05:30' - unable to parse weekday specification 'sat#000mon#000tue#000wed#000thu#000fri'#012#012
Nov 17 13:52:00 Hypervisor pvescheduler[80558]: file /etc/pve/jobs.cfg line 10 (skip section 'a0364706d6d48a82f23127bbb4e994120153cb95:2'): missing value for required option 'schedule'

Can I manually edit the config file and replace the "schedule satmontuewedthufri 05:30" of the file with something else that can be parsed? The "sat#000mon#000tue#000wed#000thu#000fri 05:30" sounds like it wants some kind of separator characters?

 

[fabian]

do you by chance know how the old vzdump.cron file looked?

and yeah, monday-saturday at 5:30 would be mon..sat 5:30 - see the 'schedule simulator' in the GUI

 

[Dunuin]

do you by chance know how the old vzdump.cron file looked?

and yeah, monday-saturday at 5:30 would be mon..sat 5:30 - see the 'schedule simulator' in the GUI

Yes, got a "/etc" backup on my PBS. Old backup job was first shown in the Webui. Then I double clicked it to edit it (it was all week days and "stop" mode originally) and I removed "sunday" in the dropdown, switched it to "snapshot" mode and clikced the OK button to save the changes. There was no error message the backup job just disappreared from the list after that.

The old vzdump.cron:

# cluster wide vzdump cron schedule
# Automatically generated file - do not edit

PATH="/usr/sbin:/usr/bin:/sbin:/bin"

30 5 * * 1           root #vzdump --exclude 123,125,103 --mailto my@mail.de --mode stop --compress zstd --mailnotification failure --all 1 --storage PBS --quiet 1
30 5 * * *           root vzdump --exclude 123,125,103 --mailto my@mail.de --mailnotification failure --mode stop --storage PBS --all 1 --quiet 1

There was one disabled job that I deleted and one enabled one that I have edited and then disappeared. So I guess the old job was "30 5 * * * root vzdump --exclude 123,125,103 --mailto my@mail.de --mailnotification failure --mode stop --storage PBS --all 1 --quiet 1".

 

[Dunuin]

Beside that problem the the PVE backup jobs...how are PVE and PBS handling pruning?

I can't see an option to disable datastore level pruning (except maybe telling PBS to keep-last=999 or something like that). So what happens if I set different pruning option on different PVE backup jobs? Does this got a higher priority than the datastore retentions on the PBS and the PVE retentions will used instead of the PBS retentions?

And what happens if I also use the proxmox-backup-client? I guess if I dont manually prune using CLI the datastores backup retentions will be used? But if I do a manual prune using CLI for a backup group this will be higher priority and prune the backup group no matter that I configured in the datastores retention?

 

[fabian]

scheduled PBS-side pruning and client (both PVE and proxmox-backup-client) side pruning are completely independent. if you set up both, the 'stricter' pruning will happen in addition to the baseline pruning.

e.g., if you setup --keep-last 100 on the PBS side and make that run daily, but then have a group that gets backed-up by PVE with a retention of --keep-last 5, --keep-daily 5, the client-side pruning will make sure that you never have more than 10 snapshots in that group (the latest 5, and then up to 5 daily on top of that).

edit: obviously the other way round doesn't make much sense, if you trim down your backups on the PBS side to a level that the client-side pruning never gets a chance to remove anything, you might as well just have the PBS side pruning

 

[dcsapak]

but I get this "Use of inherited AUTOLOAD for non-method PVE::API2::Backup::uuid() is no longer allowed at /usr/share/perl5/PVE/API2/Backup.pm line 198. (500)".

for this error, please make sure for now that you use the new gui (e.g. clear caches on the browser) there is a bug when the id (that is new and optional) is not set. the new version of the ui does send it... i already sent a fix

for the other error (the wrong conversion) i really do not have an explanation. i tried your example, but it converted just fine...

 

[Dunuin]

If I look at the "jobs.cfg" using nano instead of WinSCP I see something different:

vzdump: a0364706d6d48a82f23127bbb4e994120153cb95:2
        schedule sat^@mon^@tue^@wed^@thu^@fri 05:30
        all 1
        enabled 1
        exclude 123,125,103
        mailnotification failure
        mailto my@mail.de
        mode snapshot
        quiet 1
        storage PBS

After manually changing the line schedule sat^@mon^@tue^@wed^@thu^@fri 05:30 to schedule mon..sat 5:30 PVE WebUI will show the backup job again.

And now the GUI looks different. As I edited the backup job there was the old GUI where it only was possible to select the week days using a dropdown. Not sure if I now see the new GUI because I did a STRG+F5 after after the backup disappeared or because I edited a already existing "legacy" backup job.

 

[dcsapak]

Not sure if I now see the new GUI because I did a STRG+F5 after after the backup disappeared

thats the reason, we changed the gui and caching is sometimes weird..

also i can now reproduce the second problem too, when updating a job with the old gui, it breaks because of some weird interaction between our gui sending parameters a certain way and how we detect lists..

 

[Dunuin]

To summerize it:

1.) do a STRG+F5 before trying to edit backup jobs so you don't run into problems
2.) If you want different retention for different backup jobs set the retention using PVE GUI
3.) set the datastore retention in PBS very high (I used 999 for last, hourly, daily, ...) so that the retentions setup using PVE are actually used
4.) if you use the proxmox-backup-client to do backups also do a manual prune (so something like proxmox-backup-client backup ... && proxmox-backup-client prune ...)

 

[fabian]

3.) is not needed - if you just set client-side pruning parameters, those will be used by the client just fine without any prune scheduled on the PBS side. it might make sense to have a broad PBS-side pruning set up to catch any jobs/clients that don't do their own pruning though.