Weird sync between two PBS - wanna sync a last only

[jhr]

Hello,

I have one PBS in one location and second PBS in second location. I want to make a backup sync from time to time (let say every weekend). I want to sync all my backups from one locations to the second, but I choose to sync last only. I noticed a weird behaviour. I thought only last backup of each VM/LXC will be synced, but I saw weird behaviour. I have for example a backup of VM110 with date 2024-10-26 on second location and newest one with date 2024-11-02 on the location one, so I thought my sync process will transfere the very last one, which is 2024-11-02, but I saw date 2024-10-27 was synced (don't know why), then 2024-11-02 was synced a and then 2024-10-27 dissappeared, and of course the sync took so long (because syncing of not needed backups). I have let's say about 30 backups of VM/LXC and I have noticed that this wierd scenarion is not applying to all backups. Some of my backups are synced correctly on the first go, but some of them not. Is this behaviour have something to do with deduplications ?

THX

 

[Chris]

Hi,
please provide the task log for the sync job, that should give a bit more insight. Was snapshot 2024-11-02 already finished, or was this still an ongoing backup?

 

[jhr]

Hi,

here it is. It's one of many synced backups, this one is not so big.
2024-11-03T14:53:47+01:00: sync snapshot vm/110/2024-10-27T19:00:25Z
2024-11-03T14:53:47+01:00: sync archive qemu-server.conf.blob
2024-11-03T14:53:47+01:00: sync archive drive-virtio3.img.fidx
2024-11-03T15:04:54+01:00: downloaded 2.259 GiB (3.47 MiB/s)
2024-11-03T15:04:54+01:00: sync archive drive-virtio2.img.fidx
2024-11-03T15:06:16+01:00: downloaded 175.815 MiB (2.132 MiB/s)
2024-11-03T15:06:16+01:00: got backup log file "client.log.blob"
2024-11-03T15:06:16+01:00: sync snapshot vm/110/2024-10-27T19:00:25Z done
2024-11-03T15:06:16+01:00: percentage done: 62.50% (42/68 groups, 1/2 snapshots in group #43)
2024-11-03T15:06:16+01:00: sync snapshot vm/110/2024-11-02T19:00:23Z
2024-11-03T15:06:16+01:00: sync archive qemu-server.conf.blob
2024-11-03T15:06:16+01:00: sync archive drive-virtio3.img.fidx
2024-11-03T15:17:30+01:00: downloaded 3.289 GiB (4.999 MiB/s)
2024-11-03T15:17:30+01:00: sync archive drive-virtio2.img.fidx
2024-11-03T15:18:52+01:00: downloaded 406.001 MiB (4.993 MiB/s)
2024-11-03T15:18:52+01:00: got backup log file "client.log.blob"
2024-11-03T15:18:52+01:00: sync snapshot vm/110/2024-11-02T19:00:23Z done
2024-11-03T15:18:52+01:00: percentage done: 63.24% (43/68 groups)
2024-11-03T15:18:52+01:00: delete vanished snapshot vm/110/2024-10-26T18:00:33Z
2024-11-03T15:18:52+01:00: skipped: 2 snapshot(s) (2024-09-29T18:02:13Z .. 2024-10-20T18:02:07Z) - older than the newest local snapshot
2024-11-03T15:18:52+01:00: skipped: 5 snapshot(s) (2024-10-28T19:02:15Z .. 2024-11-01T19:02:45Z) - due to transfer-last

As you can see I did it on 2024/11/03 and the last backup was from 2024/11/02, so yes, it was finnished.

 

[Chris]

Please provide the full task log and the output of cat /etc/proxmox-backup/sync.cfg, thanks.

 

[jhr]

Here are

Thank you for analysis.

 

[Chris]

Thanks for the logs and reporting this issue.

I was able to reproduce the behavior and identified the underlying issue. Patches to fix the incorrect sync behavior and improve the filter logic have been sent to the developers mailing list [0].

[0] https://lore.proxmox.com/pbs-devel/20241104105830.85612-1-c.ebner@proxmox.com/T/