[SOLVED] bk stucked in verification

F

filippoclikkami

New Member
Dec 21, 2023
15
2
3
Hi, these days verify job got a problem with earliest backup of the set, the one that have to be deleted. Prune job is configured to maintain the 7 latest backups, it seems that it can't delete the first one and got stucked in the verification process.

I tried to prune the first one manually, run a garbage collection and reverify, everything goes right but the day after problem come back

1712652046008.png

1712652093441.png
 
please post the contents of the last verify (of that group/snapshot) and GC task log..
 
did you maybe manually try to remove that snapshot somehow? it seems a bit weird since it is the oldest, but it is in an "incomplete" state.

what does

"ls -lha /mnt/pbsbackup/vm/101/2024-03-30T20:00:02Z/"

output?
 
here the result:
root@pbsk2a:~# ls -lha /mnt/pbsbackup/vm/101/2024-04-01T19:00:02Z/
total 0
drwxr-xr-x 1 backup backup 0 Apr 10 00:11 .
drwxr-xr-x 1 backup backup 330 Apr 10 00:08 ..

yesterday snap is no longer avaible, the problem still reamin on oldest one
 
remove the empty dir, then it is gone.. I am not aware of any other reports of PBS leaving empty directories around, is it possible you tried to clear that snapshot manually?
 
yeah, that is possible (the dir has to be created before files in it can be, so if a powerloss/crash happens at a bad time then you might end up with just the empty dir).
 
is there some other task that is also started at midnight? or shortly before and still running?
 
GC doesn't lock the snapshot, but it does read its index, it's possible there is a race there. does the problem go away if you schedule the GC a few minutes later? pruning should be pretty much instant..
 
You must log in or register to reply here.
Top Bottom