vm group sync failed due to transfer-last currently written(?)

[2CsJkl8e]

Hi, a sync job: source is a 2.4-4 , target is a 3.1-2 , transfer-last = 1 , nothing special otherwise.
Tonight the sync job failed with:
2024-01-31T19:16:25+00:00: sync group vm/166 failed - unable to acquire lock on snapshot directory "/mnt/datastore/123/ns/123/vm/166/2024-01-31T19:15:53Z" - locked by another operation
Obviously, this particular snapshot was currently in writing on source, backup start: 19:05:59Z - end: 19:16:51Z.
Everything consistent so far.
The question is however, why do you select a snapshot as transfer-last if it is deliberate under wrinting (and locked) ?? ))
Did I miss something?
Thanks Peter

 

[fabian]

there is a window where the backup already shows up as "finished", but is still locked for post-processing. but your time line doesn't really line up - the backup "2024-01-31T19:15:53Z" cannot have started at 19:05:59Z unless your systems are really out of sync or you manually do backups with "fake" timestamps

 

[2CsJkl8e]

Hi, aha, understand. i sent you the backup start time according to the pve vm task log timestamps, see below.
but true, within the task log, we find:
INFO: Backup started at 2024-01-31 20:15:53
INFO: Backup finished at 2024-01-31 20:16:51
Nevertheless, the failure time as of 19:16:25+00:00 is still within the backup time I guess.
What now?
Thanks Peter

 

[fabian]

check the backup task log on the PBS side for that particular snapshot - it was probably still running and still holding the lock, or you have a prune-after-backup setup where the sync and the prune mis-aligned (which should also be visible on the PBS side if you look at the tasks and timestamps). in any case, the sync did what it was supposed to do - it tried to sync that snapshot, but failed because it was locked (either still, or already again) - and thus skipped that snapshot and told you about it..

 

[2CsJkl8e]

feature request: on a "transfer-last = 1" sync, behave smarter, such that a snapshot still running remains untouched and the next younger (finished) one (if existing) is transfered

 

[fabian]

the problem is that from the client (and sync is a client here) perspective the backup is finished, we can't tell until we start the reader that it's not. see https://bugzilla.proxmox.com/show_bug.cgi?id=4523 for the nitty gritty details.