[SOLVED] proxmox-backup-client bug

[mtxct]

Hi,

First time post here :
I detected a bug on PBS 2.4-1, I'm not sure if I should post it here.

==============================================================

I have some VMs backuped every day. For instance : ID 100
and some VMs backuped once a week. For instance : ID 113

same storage and namespace

for a prune job like this : --keep-daily 7 --keep-weekly 4 --keep-monthly 3

In VM 100 :
┌─────────────────────────────┬──────────────────────────┬────────┐
│ snapshot │ date │ action │
╞═════════════════════════════╪══════════════════════════╪════════╡
│ vm/100/2023-02-26T00:00:06Z │ Sun Feb 26 01:00:06 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/100/2023-03-26T00:00:00Z │ Sun Mar 26 01:00:00 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/100/2023-04-01T23:00:00Z │ Sun Apr 2 01:00:00 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/100/2023-04-08T23:00:01Z │ Sun Apr 9 01:00:01 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/100/2023-04-15T23:00:02Z │ Sun Apr 16 01:00:02 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/100/2023-04-22T23:00:02Z │ Sun Apr 23 01:00:02 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/100/2023-04-24T20:00:00Z │ Mon Apr 24 22:00:00 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/100/2023-04-25T20:00:06Z │ Tue Apr 25 22:00:06 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/100/2023-04-26T20:00:08Z │ Wed Apr 26 22:00:08 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/100/2023-04-27T20:00:04Z │ Thu Apr 27 22:00:04 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/100/2023-04-28T20:00:03Z │ Fri Apr 28 22:00:03 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/100/2023-04-29T23:00:00Z │ Sun Apr 30 01:00:00 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/100/2023-05-01T20:00:00Z │ Mon May 1 22:00:00 2023 │ keep │
└─────────────────────────────┴──────────────────────────┴────────┘

In VM 113 :
┌─────────────────────────────┬──────────────────────────┬────────┐
│ snapshot │ date │ action │
╞═════════════════════════════╪══════════════════════════╪════════╡
│ vm/113/2023-02-19T02:07:15Z │ Sun Feb 19 03:07:15 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/113/2023-02-26T00:10:25Z │ Sun Feb 26 01:10:25 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/113/2023-03-05T00:07:23Z │ Sun Mar 5 01:07:23 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/113/2023-03-12T00:07:33Z │ Sun Mar 12 01:07:33 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/113/2023-03-19T00:08:35Z │ Sun Mar 19 01:08:35 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/113/2023-03-26T00:07:05Z │ Sun Mar 26 01:07:05 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/113/2023-04-01T23:06:53Z │ Sun Apr 2 01:06:53 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/113/2023-04-08T23:08:58Z │ Sun Apr 9 01:08:58 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/113/2023-04-15T23:07:58Z │ Sun Apr 16 01:07:58 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/113/2023-04-22T23:06:52Z │ Sun Apr 23 01:06:52 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/113/2023-04-29T23:06:52Z │ Sun Apr 30 01:06:52 2023 │ keep │
└─────────────────────────────┴──────────────────────────┴────────┘

If I change to --keep-daily 2 --keep-weekly 4 --keep-monthly 3 :

For VM 100 :
┌─────────────────────────────┬──────────────────────────┬────────┐
│ snapshot │ date │ action │
╞═════════════════════════════╪══════════════════════════╪════════╡
│ vm/100/2023-02-26T00:00:06Z │ Sun Feb 26 01:00:06 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/100/2023-03-26T00:00:00Z │ Sun Mar 26 01:00:00 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/100/2023-04-01T23:00:00Z │ Sun Apr 2 01:00:00 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/100/2023-04-08T23:00:01Z │ Sun Apr 9 01:00:01 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/100/2023-04-15T23:00:02Z │ Sun Apr 16 01:00:02 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/100/2023-04-22T23:00:02Z │ Sun Apr 23 01:00:02 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/100/2023-04-24T20:00:00Z │ Mon Apr 24 22:00:00 2023 │ remove │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/100/2023-04-25T20:00:06Z │ Tue Apr 25 22:00:06 2023 │ remove │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/100/2023-04-26T20:00:08Z │ Wed Apr 26 22:00:08 2023 │ remove │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/100/2023-04-27T20:00:04Z │ Thu Apr 27 22:00:04 2023 │ remove │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/100/2023-04-28T20:00:03Z │ Fri Apr 28 22:00:03 2023 │ remove │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/100/2023-04-29T23:00:00Z │ Sun Apr 30 01:00:00 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/100/2023-05-01T20:00:00Z │ Mon May 1 22:00:00 2023 │ keep │
└─────────────────────────────┴──────────────────────────┴────────┘

For VM 113 :
┌─────────────────────────────┬──────────────────────────┬────────┐
│ snapshot │ date │ action │
╞═════════════════════════════╪══════════════════════════╪════════╡
│ vm/113/2023-02-19T02:07:15Z │ Sun Feb 19 03:07:15 2023 │ remove │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/113/2023-02-26T00:10:25Z │ Sun Feb 26 01:10:25 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/113/2023-03-05T00:07:23Z │ Sun Mar 5 01:07:23 2023 │ remove │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/113/2023-03-12T00:07:33Z │ Sun Mar 12 01:07:33 2023 │ remove │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/113/2023-03-19T00:08:35Z │ Sun Mar 19 01:08:35 2023 │ remove │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/113/2023-03-26T00:07:05Z │ Sun Mar 26 01:07:05 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/113/2023-04-01T23:06:53Z │ Sun Apr 2 01:06:53 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/113/2023-04-08T23:08:58Z │ Sun Apr 9 01:08:58 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/113/2023-04-15T23:07:58Z │ Sun Apr 16 01:07:58 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/113/2023-04-22T23:06:52Z │ Sun Apr 23 01:06:52 2023 │ keep │
├─────────────────────────────┼──────────────────────────┼────────┤
│ vm/113/2023-04-29T23:06:52Z │ Sun Apr 30 01:06:52 2023 │ keep │
└─────────────────────────────┴──────────────────────────┴────────┘

As you can see, the dates in VM 113 are all wrong.
So my guess is when there are no daily backups, the option --keep-daily behaves as --keep-last : keeping the last 7 backups + the last 4 weeks and last 3 months... but the expected behavior is to just keep the last 4 weeks and last 3 months because there are no daily backups

Thanks

 

[sterzy]

Hi!

Proxmox Backup Server doesn't really know how often a backup is made, since that is usually specified on the client (Proxmox VE) side. So keep-daily really means something like “keep 7 backups while only keeping the last one of any given day”. Does that make sense?

You may also want to check out the prune simulator[1] to get a better understanding/visualization of this.

[1]: https://pbs.proxmox.com/docs/prune-simulator/index.html

 

[mtxct]

Hi!
Thanks for your quick response time! I'm amazed !!

I understand your answer but this is not what is happening in my humble opinion.

In the last table I posted (VM 113 --keep-daily 2 --keep-weekly 4 --keep-monthly 3 ), why it is keeping the last 6 weeks of backup ? This is not correct right ? It should only keep 4 weeks.

Keep daily 2 shouldn't apply because there are no 2 days consecutive backups. All the backups are 1 week apart.

That is the bug I'm reporting.

Thanks

 

[sterzy]

I understand that this may be a little odd, but it is consistent with how we apply these rules. From the documentation:

--keep-daily <N>
Keep backups for the last <N> days. If there is more than one backup for a single day, only the latest is kept.

Note that it does not specify “consecutive” days. Also, this:

The retention options are processed in the order given above. Each option only covers backups within its time period. The next option does not take care of already covered backups. It will only consider older backups.

So your command essentially reads as such:

• --keep-daily 2: Keep backups from the last 2 (non-consecutive) days, if there is more than one backup in a day, keep the latest.
• --keep-weekly 4: Disregarding all previously covered backups, keep 4 backups per (non-consecutive) week, if there is more than one, keep the latest.
• --keep-monthly 3: Disregarding all previously covered backups, keep 2 backups per (non-consecutive) month, if there is more than one, keep the latest.

So keep-daily will keep the backups from the last two Sundays, keep-weekly wil keep another four backups from the last four weeks, and keep-monthly will keep an additional 3 from the last three months.

If you made the rules apply only to consecutive days you will run into problems. For example, what happens if a backup falls through for a day? Say you have a backup for Thursday, Tuesday and Monday, but Wednesday you had an outage. Should keep-daily 2 prune Monday? The current rules would say yes, because that would consider Thursday and Tuesday as the last two backups. If you only apply it to consecutive days, then they wouldn't count. Now you are in a bit of a conundrum. Should you remove Monday even though there aren't two consecutive daily backups after it?[/icode]

 

[mtxct]

Hi,

Thanks for the explanation. I find it a little odd but OK, as long ass this behavior is intended I can plan around those rules.

Thanks to you and all the team that makes proxmox possible!

 

[sterzy]

No problem and thank you for using our products! Please mark this thread as solved by clicking the “Edit Thread” button at the top of the page and selecting the “Solved” prefix. That will help others find this thread more easily if they have a similar question. Thanks!