No space in backup server - help

[deepcloud]

2024-09-07T11:27:11+05:30: starting garbage collection on store backup01
2024-09-07T11:27:11+05:30: Start GC phase1 (mark used chunks)
2024-09-07T11:27:11+05:30: TASK ERROR: update atime failed for chunk/file "/backup01/.chunks/acd5/acd52f1765fbeaa474456d2fd039a9987764f7bca37af1d94f0fa1922cbf00c1" - ENOSPC: No space left on device

even deleting a few backups does not reclaim space

reduced the number of hourly backups to be retained from 8 to 3 and I did a prun still no luck

gc collection fails with the above error. and space continues to be at zero bytes available.

 

[sw-omit]

Garbage collection does indeed need some free space to function.
If I recall correctly the tip I've seen used before (still looking for the reference, might post it if I do find it) is to take some of the chunks from the .chunks folder, move it to someplace else, then run the GC. It will give some warnings/errors about the missing chunks, but should clean up some data. After that, place the moved chunks back and run a verify/GC again.

 

[sw-omit]

Found at least two references:
https://forum.proxmox.com/threads/unable-to-delete-backups-no-space-left-on-device.141218/
https://forum.proxmox.com/threads/how-to-recover-from-100-disk-use.142044/#post-637174