garbage collector seems to not work

[fanfan2a]

Hi

I have installed PBS as a VM on my PC. The VM is launched every day at 11PM, and stopped automatically at 2AM (for security reasons)

in the history, I can see that garbage collector is launched every day around 00:00 . I can see pending removals every day but the files are never deleted from the disk (I use a NFS storage on QNAP)

I've tried with the VM powered 24/24 but I have the same issue

An idea of the problem?

Thanks All

 

[fabian]

please post the full GC task log

 

[fanfan2a]

here is the last one :

[CODE]2023-10-17T00:00:00+02:00: starting garbage collection on store QNAP 2023-10-17T00:00:00+02:00: task triggered by schedule 'daily' 2023-10-17T00:00:00+02:00: Start GC phase1 (mark used chunks) 2023-10-17T00:01:41+02:00: marked 9% (1 of 11 index files) 2023-10-17T00:02:04+02:00: marked 18% (2 of 11 index files) 2023-10-17T00:02:51+02:00: marked 27% (3 of 11 index files) 2023-10-17T00:02:57+02:00: marked 36% (4 of 11 index files) 2023-10-17T00:02:57+02:00: marked 45% (5 of 11 index files) 2023-10-17T00:03:35+02:00: marked 54% (6 of 11 index files) 2023-10-17T00:03:35+02:00: marked 63% (7 of 11 index files) 2023-10-17T00:03:41+02:00: marked 72% (8 of 11 index files) 2023-10-17T00:04:19+02:00: marked 81% (9 of 11 index files) 2023-10-17T00:04:19+02:00: marked 90% (10 of 11 index files) 2023-10-17T00:04:25+02:00: marked 100% (11 of 11 index files) 2023-10-17T00:04:25+02:00: Start GC phase2 (sweep unused chunks) 2023-10-17T00:04:26+02:00: processed 1% (299 chunks) 2023-10-17T00:04:28+02:00: processed 2% (562 chunks) 2023-10-17T00:04:29+02:00: processed 3% (851 chunks) 2023-10-17T00:04:30+02:00: processed 4% (1128 chunks) 2023-10-17T00:04:32+02:00: processed 5% (1381 chunks) 2023-10-17T00:04:34+02:00: processed 6% (1676 chunks) 2023-10-17T00:04:36+02:00: processed 7% (1959 chunks) 2023-10-17T00:04:37+02:00: processed 8% (2215 chunks) 2023-10-17T00:04:39+02:00: processed 9% (2482 chunks) 2023-10-17T00:04:41+02:00: processed 10% (2767 chunks) 2023-10-17T00:04:42+02:00: processed 11% (3074 chunks) 2023-10-17T00:04:44+02:00: processed 12% (3351 chunks) 2023-10-17T00:04:45+02:00: processed 13% (3617 chunks) 2023-10-17T00:04:47+02:00: processed 14% (3896 chunks) 2023-10-17T00:04:49+02:00: processed 15% (4154 chunks) 2023-10-17T00:04:51+02:00: processed 16% (4426 chunks) 2023-10-17T00:04:52+02:00: processed 17% (4712 chunks) 2023-10-17T00:04:54+02:00: processed 18% (5008 chunks) 2023-10-17T00:04:56+02:00: processed 19% (5270 chunks) 2023-10-17T00:04:58+02:00: processed 20% (5554 chunks) 2023-10-17T00:04:59+02:00: processed 21% (5827 chunks) 2023-10-17T00:05:01+02:00: processed 22% (6136 chunks) 2023-10-17T00:05:03+02:00: processed 23% (6394 chunks) 2023-10-17T00:05:06+02:00: processed 24% (6678 chunks) 2023-10-17T00:05:10+02:00: processed 25% (6970 chunks) 2023-10-17T00:05:12+02:00: processed 26% (7240 chunks) 2023-10-17T00:05:14+02:00: processed 27% (7505 chunks) 2023-10-17T00:05:15+02:00: processed 28% (7806 chunks) 2023-10-17T00:05:17+02:00: processed 29% (8075 chunks) 2023-10-17T00:05:19+02:00: processed 30% (8338 chunks) 2023-10-17T00:05:20+02:00: processed 31% (8611 chunks) 2023-10-17T00:05:22+02:00: processed 32% (8881 chunks) 2023-10-17T00:05:23+02:00: processed 33% (9194 chunks) 2023-10-17T00:05:25+02:00: processed 34% (9464 chunks) 2023-10-17T00:05:27+02:00: processed 35% (9709 chunks) 2023-10-17T00:05:29+02:00: processed 36% (9986 chunks) 2023-10-17T00:05:30+02:00: processed 37% (10276 chunks) 2023-10-17T00:05:32+02:00: processed 38% (10571 chunks) 2023-10-17T00:05:33+02:00: processed 39% (10873 chunks) 2023-10-17T00:05:35+02:00: processed 40% (11127 chunks) 2023-10-17T00:05:37+02:00: processed 41% (11390 chunks) 2023-10-17T00:05:39+02:00: processed 42% (11663 chunks) 2023-10-17T00:05:40+02:00: processed 43% (11918 chunks) 2023-10-17T00:05:42+02:00: processed 44% (12202 chunks) 2023-10-17T00:05:44+02:00: processed 45% (12479 chunks) 2023-10-17T00:05:45+02:00: processed 46% (12751 chunks) 2023-10-17T00:05:47+02:00: processed 47% (13008 chunks) 2023-10-17T00:05:49+02:00: processed 48% (13303 chunks) 2023-10-17T00:05:51+02:00: processed 49% (13570 chunks) 2023-10-17T00:05:53+02:00: processed 50% (13855 chunks) 2023-10-17T00:05:55+02:00: processed 51% (14130 chunks) 2023-10-17T00:05:57+02:00: processed 52% (14385 chunks) 2023-10-17T00:05:59+02:00: processed 53% (14671 chunks) 2023-10-17T00:06:00+02:00: processed 54% (14945 chunks) 2023-10-17T00:06:03+02:00: processed 55% (15219 chunks) 2023-10-17T00:06:04+02:00: processed 56% (15500 chunks) 2023-10-17T00:06:06+02:00: processed 57% (15771 chunks) 2023-10-17T00:06:07+02:00: processed 58% (16060 chunks) 2023-10-17T00:06:09+02:00: processed 59% (16337 chunks) 2023-10-17T00:06:11+02:00: processed 60% (16607 chunks) 2023-10-17T00:06:13+02:00: processed 61% (16878 chunks) 2023-10-17T00:06:14+02:00: processed 62% (17188 chunks) 2023-10-17T00:06:16+02:00: processed 63% (17483 chunks) 2023-10-17T00:06:18+02:00: processed 64% (17765 chunks) 2023-10-17T00:06:20+02:00: processed 65% (18044 chunks) 2023-10-17T00:06:22+02:00: processed 66% (18326 chunks) 2023-10-17T00:06:24+02:00: processed 67% (18586 chunks) 2023-10-17T00:06:25+02:00: processed 68% (18888 chunks) 2023-10-17T00:06:27+02:00: processed 69% (19133 chunks) 2023-10-17T00:06:29+02:00: processed 70% (19380 chunks) 2023-10-17T00:06:30+02:00: processed 71% (19676 chunks) 2023-10-17T00:06:32+02:00: processed 72% (19969 chunks) 2023-10-17T00:06:34+02:00: processed 73% (20252 chunks) 2023-10-17T00:06:36+02:00: processed 74% (20516 chunks) 2023-10-17T00:06:37+02:00: processed 75% (20806 chunks) 2023-10-17T00:06:39+02:00: processed 76% (21089 chunks) 2023-10-17T00:06:41+02:00: processed 77% (21360 chunks) 2023-10-17T00:06:43+02:00: processed 78% (21624 chunks) 2023-10-17T00:06:44+02:00: processed 79% (21929 chunks) 2023-10-17T00:06:46+02:00: processed 80% (22227 chunks) 2023-10-17T00:06:48+02:00: processed 81% (22501 chunks) 2023-10-17T00:06:49+02:00: processed 82% (22748 chunks) 2023-10-17T00:06:51+02:00: processed 83% (23034 chunks) 2023-10-17T00:06:52+02:00: processed 84% (23317 chunks) 2023-10-17T00:06:54+02:00: processed 85% (23605 chunks) 2023-10-17T00:06:56+02:00: processed 86% (23883 chunks) 2023-10-17T00:06:58+02:00: processed 87% (24171 chunks) 2023-10-17T00:07:00+02:00: processed 88% (24424 chunks) 2023-10-17T00:07:01+02:00: processed 89% (24658 chunks) 2023-10-17T00:07:03+02:00: processed 90% (24938 chunks) 2023-10-17T00:07:05+02:00: processed 91% (25242 chunks) 2023-10-17T00:07:06+02:00: processed 92% (25530 chunks) 2023-10-17T00:07:08+02:00: processed 93% (25824 chunks) 2023-10-17T00:07:10+02:00: processed 94% (26095 chunks) 2023-10-17T00:07:12+02:00: processed 95% (26382 chunks) 2023-10-17T00:07:14+02:00: processed 96% (26692 chunks) 2023-10-17T00:07:16+02:00: processed 97% (26984 chunks) 2023-10-17T00:07:17+02:00: processed 98% (27240 chunks) 2023-10-17T00:07:19+02:00: processed 99% (27517 chunks) 2023-10-17T00:07:21+02:00: Removed garbage: 201.251 MiB 2023-10-17T00:07:21+02:00: Removed chunks: 155 2023-10-17T00:07:21+02:00: Pending removals: 201.742 MiB (in 367 chunks) 2023-10-17T00:07:21+02:00: Original data usage: 320 GiB 2023-10-17T00:07:21+02:00: On-Disk usage: 60.689 GiB (18.97%) 2023-10-17T00:07:21+02:00: On-Disk chunks: 27267 2023-10-17T00:07:21+02:00: Deduplication factor: 5.27 2023-10-17T00:07:21+02:00: Average chunk size: 2.279 MiB 2023-10-17T00:07:21+02:00: TASK OK[/CODE]

 

[fabian]

so it removed some chunks, and some more are marked as pending (this means the chunks are not referenced, but are after the cutoff for GC, so they might be new chunks of currently ongoing backups, or chunks of fairly fresh snapshots that were already pruned), which looks okay.

how are your prune settings?

 

[fanfan2a]

retention on PVE :

prune on PBS :

 

[fabian]

as long as the pending removals don't go up while the removals are zero, you should be fine. you are both pruning rather aggressively and often, and run GC quite often, so the amount removed by each GC will not be much (it will basically amount to the amount of changed data per day, but with a bit of delay).