Backups are skipped when syncing PBS to another PBS

[powersupport]

Hi,

When we try syncing PBS to another PBS some of the backups are skipped. Please refer to the attached warning.

==
2022-08-02T16:47:30+05:30: skipped: 73 snapshot(s) (2021-12-26T17:30:02Z .. 2022-07-31T13:30:02Z) older than the newest local snapshot
==

May I know if there is any way to resolve this issue?

Please advise.

Regards,

 

[fabian]

this is currently not possible, the sync uses the last local timestamp to determine the cut-off for syncing. if you want to sync all snapshots, you can sync into a namespace not containing that group (it will still share the chunk store, so already existing chunks won't need to be re-downloaded).

 

[cheesecake]

@fabian, is there any news when this will be possible? it would be really great with this feature, gladly configurable too.

best cheeseck

 

[fabian]

no ETA at the moment. initial patches were written, the bug to track progress is https://bugzilla.proxmox.com/show_bug.cgi?id=3786 (there are two aspects mentioned there - both syncing "gaps"/missing snapshots, and re-syncing known-corrupt snapshots from a hopefully still good remote)

 

[santacruzskim]

+1 here.
I recently set up an offsite backup server ("pbs2") to sync with my home backup server ("pbs1") and on my first sync with "pbs1" I choose to only sync the most recent snapshots via the "transfer last <n>" option. I did this to only transfer a subset of data first to confirm the connection, monitor transfer speed, make sure nothing horrible was happening, etc. before sending off a full sync of multiple TB of data. The "transfer last 1' job ran successfully, but when I flipped it to "last 3" or even "all" it would just report that all other backups were older than the latest (which had already been transferred earlier) so they were skipped.

I came across a workaround used in similar scenarios of creating a new namespace in the destination (pbs2) datastore and editing the sync job's "Local Namespace" to point to the newly created one and it's running now, backing up the remaining chunks.

This is obviously not ideal. If I say I want "all" backups, it shouldn't override that simply because there are newer backups - in the world of backups, newer isn't always better - and using namepaces in this way is misusing the feature to work around the real problem.

Hopefully this use case will help guide any future development in patching this (tiny) hole in this great product.