[SOLVED] Retention in a sync job

[Fra]

We have two Proxmox Backup Servers:

* the main one (the Remote-pbs), in the internet

* the second one, on premises, that has a daily sync job with the Remote: I call it Local-pbs


In the Local-pbs the retention is "lower" than the retention in the main/Remote-pbs one (so this Remote-pbs has many more snapshots than the Local-pbs on premises, since we are happy with just few latest backups: it is just an 'emergency' backup of the backup already downloaded on premises: just in case...).

Yesterday I pruned the Local-pbs: then today I started the daily sync job, and to me it looks like the sync job is 'copying' locally snapshots that will be removed on the next prune: is this by design? I mean, does the sync job with the Remote sync everything, ignoring the 'local' retention (which could be 'lower')?

As for the 'lower' retention (forgive me the improper term), to simplify, let's imagine we only set the keep-daily, and the Remote-pbs has a keep-daily:10, and the Local-pbs a keep-daily:3.

(I ask this since our local-pbs does not has high speed internet, so our sync job last the whole day, then the GC another 2 hours, so avoiding useless sync/prune/removal of snapshots would help a lot)

 

[dcsapak]

I mean, does the sync job with the Remote sync everything, ignoring the 'local' retention (which could be 'lower')?

yes because the sync job is independent from the prune settings.

i guess it would be possible to take those into account, but that may have other unintended side effects, since we'd either have to prune local backups too if the settings would result in that, or have a
'in-between' state where the content does not represent the prune settings
(this could happen if there are also regular backups happen on this datastore and not only syncs)

you could open a feature request here: https://bugzilla.proxmox.com that is the better place to discuss such things

 

[Fra]

yes because the sync job is independent from the prune settings.

oh, I see

that's another reason why our local GC takes ages every day

you could open a feature request here: https://bugzilla.proxmox.com that is the better place to discuss such things

Ok, I did
https://bugzilla.proxmox.com/show_bug.cgi?id=3977

 

[Fra]

Btw, I read this in the rsyc-remote job but I am not sure:

2022-04-02T08:38:31+02:00: skipped: 18 snapshot(s) (2021-12-26T22:50:01Z
 .. 2022-03-31T22:07:11Z) older than the newest local snapshot

it comes from the full log below:

2022-04-02T08:38:30+02:00: re-sync snapshot "vm/908/2022-04-01T21:39:33Z"
2022-04-02T08:38:30+02:00: no data changes
2022-04-02T08:38:30+02:00: re-sync snapshot "vm/908/2022-04-01T21:39:33Z" done
2022-04-02T08:38:30+02:00: percentage done: 98.00% (49/50 groups)
2022-04-02T08:38:30+02:00: delete vanished snapshot "vm/908/2022-03-28T22:08:53Z"
2022-04-02T08:38:30+02:00: skipped: 18 snapshot(s) (2021-12-26T22:48:44Z .. 2022-03-31T22:05:55Z) older than the newest local snapshot
2022-04-02T08:38:31+02:00: re-sync snapshot "vm/999/2022-04-01T21:42:43Z"
2022-04-02T08:38:31+02:00: no data changes
2022-04-02T08:38:31+02:00: re-sync snapshot "vm/999/2022-04-01T21:42:43Z" done
2022-04-02T08:38:31+02:00: percentage done: 100.00% (50/50 groups)
2022-04-02T08:38:31+02:00: delete vanished snapshot "vm/999/2022-03-28T22:10:06Z"
2022-04-02T08:38:31+02:00: skipped: 18 snapshot(s) (2021-12-26T22:50:01Z .. 2022-03-31T22:07:11Z) older than the newest local snapshot

If the job skip old snapshots, why it did not skip the "vm/908/2022-04-01T21:39:33Z" on the first line, too? because is the newest local?

 

[dcsapak]

we try to resync exising snapshots (for comments for example), but we do not sync *new* snapshots that are older than the newest locally

 

[Fra]

we try to resync exising snapshots (for comments for example), but we do not sync *new* snapshots that are older than the newest locally

ah, ok! those asterisks around the *new* helped! thank!



Btw, if anybody needs more on this: my feature request in bugzilla was not needed since it was already there https://bugzilla.proxmox.com/show_bug.cgi?id=3977 and this thread https://forum.proxmox.com/threads/partial-sync-job-pre-prune.106441/ helped me too.

 

[LordRatner]

If a backup was pruned previously from a remote, will it re-sync later only to be pruned again?

So the main PBS holds the last 10 backups. The remote prunes all but the last three. Will the remote sync all ten every time, only to prune 7 of them?

 

[dcsapak]

So the main PBS holds the last 10 backups. The remote prunes all but the last three. Will the remote sync all ten every time, only to prune 7 of them?

as i wrote above:

we try to resync exising snapshots (for comments for example), but we do not sync *new* snapshots that are older than the newest locally