Hi folks,
I have read some threads about GC here in forum, but was not able to get any answer to my question/issue
for me it seems Garbage Collect takes quite long.
I have PC where I am running Proxmox VE and on this PVE I have 3 VMs:
1. Docker (Ubuntu)
2. Home assistant
3. PBS
On PBS I have scheduled daily backups of first 2 mentioned VMs and storage is NFS share on my Synology NAS
Garbage Collect log (job takes more than 2 hours)
Backup takes few minutes - 2-3 minutes
Is ths normal? Or how long the GC prune should take?
Thanks
I have read some threads about GC here in forum, but was not able to get any answer to my question/issue
for me it seems Garbage Collect takes quite long.
I have PC where I am running Proxmox VE and on this PVE I have 3 VMs:
1. Docker (Ubuntu)
2. Home assistant
3. PBS
On PBS I have scheduled daily backups of first 2 mentioned VMs and storage is NFS share on my Synology NAS
Garbage Collect log (job takes more than 2 hours)
Code:
2024-10-02T00:00:00+02:00: starting garbage collection on store Backup_proxmox
2024-10-02T00:00:00+02:00: task triggered by schedule 'daily'
2024-10-02T00:00:00+02:00: Start GC phase1 (mark used chunks)
2024-10-02T00:04:47+02:00: marked 1% (2 of 162 index files)
2024-10-02T00:07:43+02:00: marked 2% (4 of 162 index files)
2024-10-02T00:09:53+02:00: marked 3% (5 of 162 index files)
2024-10-02T00:11:10+02:00: marked 4% (7 of 162 index files)
2024-10-02T00:12:33+02:00: marked 5% (9 of 162 index files)
2024-10-02T00:13:14+02:00: marked 6% (10 of 162 index files)
2024-10-02T00:13:56+02:00: marked 7% (12 of 162 index files)
2024-10-02T00:13:57+02:00: marked 8% (13 of 162 index files)
2024-10-02T00:14:39+02:00: marked 9% (15 of 162 index files)
2024-10-02T00:14:39+02:00: marked 10% (17 of 162 index files)
2024-10-02T00:15:39+02:00: marked 11% (18 of 162 index files)
2024-10-02T00:16:16+02:00: marked 12% (20 of 162 index files)
2024-10-02T00:18:03+02:00: marked 13% (22 of 162 index files)
2024-10-02T00:19:40+02:00: marked 14% (23 of 162 index files)
2024-10-02T00:22:41+02:00: marked 15% (25 of 162 index files)
2024-10-02T00:24:12+02:00: marked 16% (26 of 162 index files)
2024-10-02T00:27:04+02:00: marked 17% (28 of 162 index files)
2024-10-02T00:30:04+02:00: marked 18% (30 of 162 index files)
2024-10-02T00:31:45+02:00: marked 19% (31 of 162 index files)
2024-10-02T00:36:18+02:00: marked 20% (33 of 162 index files)
2024-10-02T00:40:05+02:00: marked 21% (35 of 162 index files)
2024-10-02T00:41:38+02:00: marked 22% (36 of 162 index files)
2024-10-02T00:46:44+02:00: marked 23% (38 of 162 index files)
2024-10-02T00:48:46+02:00: marked 24% (39 of 162 index files)
2024-10-02T00:51:26+02:00: marked 25% (41 of 162 index files)
2024-10-02T00:54:00+02:00: marked 26% (43 of 162 index files)
2024-10-02T00:55:11+02:00: marked 27% (44 of 162 index files)
2024-10-02T00:58:04+02:00: marked 28% (46 of 162 index files)
2024-10-02T00:59:08+02:00: marked 29% (47 of 162 index files)
2024-10-02T01:02:24+02:00: marked 30% (49 of 162 index files)
2024-10-02T01:05:44+02:00: marked 31% (51 of 162 index files)
2024-10-02T01:07:08+02:00: marked 32% (52 of 162 index files)
2024-10-02T01:10:17+02:00: marked 33% (54 of 162 index files)
2024-10-02T01:13:01+02:00: marked 34% (56 of 162 index files)
2024-10-02T01:14:10+02:00: marked 35% (57 of 162 index files)
2024-10-02T01:16:30+02:00: marked 36% (59 of 162 index files)
2024-10-02T01:17:38+02:00: marked 37% (60 of 162 index files)
2024-10-02T01:20:01+02:00: marked 38% (62 of 162 index files)
2024-10-02T01:22:27+02:00: marked 39% (64 of 162 index files)
2024-10-02T01:23:38+02:00: marked 40% (65 of 162 index files)
2024-10-02T01:26:52+02:00: marked 41% (67 of 162 index files)
2024-10-02T01:29:37+02:00: marked 42% (69 of 162 index files)
2024-10-02T01:29:37+02:00: marked 43% (70 of 162 index files)
2024-10-02T01:30:18+02:00: marked 44% (72 of 162 index files)
2024-10-02T01:30:47+02:00: marked 45% (73 of 162 index files)
2024-10-02T01:31:58+02:00: marked 46% (75 of 162 index files)
2024-10-02T01:32:35+02:00: marked 47% (77 of 162 index files)
2024-10-02T01:32:35+02:00: marked 48% (78 of 162 index files)
2024-10-02T01:33:11+02:00: marked 49% (80 of 162 index files)
2024-10-02T01:33:45+02:00: marked 50% (81 of 162 index files)
2024-10-02T01:34:38+02:00: marked 51% (83 of 162 index files)
2024-10-02T01:35:28+02:00: marked 52% (85 of 162 index files)
2024-10-02T01:35:28+02:00: marked 53% (86 of 162 index files)
2024-10-02T01:36:10+02:00: marked 54% (88 of 162 index files)
2024-10-02T01:36:50+02:00: marked 55% (90 of 162 index files)
2024-10-02T01:37:28+02:00: marked 56% (91 of 162 index files)
2024-10-02T01:38:07+02:00: marked 57% (93 of 162 index files)
2024-10-02T01:38:07+02:00: marked 58% (94 of 162 index files)
2024-10-02T01:38:47+02:00: marked 59% (96 of 162 index files)
2024-10-02T01:39:25+02:00: marked 60% (98 of 162 index files)
2024-10-02T01:40:14+02:00: marked 61% (99 of 162 index files)
2024-10-02T01:40:42+02:00: marked 62% (101 of 162 index files)
2024-10-02T01:41:20+02:00: marked 63% (103 of 162 index files)
2024-10-02T01:41:20+02:00: marked 64% (104 of 162 index files)
2024-10-02T01:41:54+02:00: marked 65% (106 of 162 index files)
2024-10-02T01:42:36+02:00: marked 66% (107 of 162 index files)
2024-10-02T01:43:32+02:00: marked 67% (109 of 162 index files)
2024-10-02T01:44:09+02:00: marked 68% (111 of 162 index files)
2024-10-02T01:44:09+02:00: marked 69% (112 of 162 index files)
2024-10-02T01:44:55+02:00: marked 70% (114 of 162 index files)
2024-10-02T01:45:53+02:00: marked 71% (116 of 162 index files)
2024-10-02T01:46:32+02:00: marked 72% (117 of 162 index files)
2024-10-02T01:47:05+02:00: marked 73% (119 of 162 index files)
2024-10-02T01:47:05+02:00: marked 74% (120 of 162 index files)
2024-10-02T01:47:38+02:00: marked 75% (122 of 162 index files)
2024-10-02T01:48:18+02:00: marked 76% (124 of 162 index files)
2024-10-02T01:48:56+02:00: marked 77% (125 of 162 index files)
2024-10-02T01:49:34+02:00: marked 78% (127 of 162 index files)
2024-10-02T01:49:34+02:00: marked 79% (128 of 162 index files)
2024-10-02T01:50:19+02:00: marked 80% (130 of 162 index files)
2024-10-02T01:50:59+02:00: marked 81% (132 of 162 index files)
2024-10-02T01:51:32+02:00: marked 82% (133 of 162 index files)
2024-10-02T01:52:12+02:00: marked 83% (135 of 162 index files)
2024-10-02T01:52:44+02:00: marked 84% (137 of 162 index files)
2024-10-02T01:52:44+02:00: marked 85% (138 of 162 index files)
2024-10-02T01:53:24+02:00: marked 86% (140 of 162 index files)
2024-10-02T01:54:02+02:00: marked 87% (141 of 162 index files)
2024-10-02T01:54:39+02:00: marked 88% (143 of 162 index files)
2024-10-02T01:55:13+02:00: marked 89% (145 of 162 index files)
2024-10-02T01:55:13+02:00: marked 90% (146 of 162 index files)
2024-10-02T01:55:50+02:00: marked 91% (148 of 162 index files)
2024-10-02T01:56:42+02:00: marked 92% (150 of 162 index files)
2024-10-02T01:57:11+02:00: marked 93% (151 of 162 index files)
2024-10-02T01:57:52+02:00: marked 94% (153 of 162 index files)
2024-10-02T01:57:52+02:00: marked 95% (154 of 162 index files)
2024-10-02T01:58:33+02:00: marked 96% (156 of 162 index files)
2024-10-02T01:59:09+02:00: marked 97% (158 of 162 index files)
2024-10-02T01:59:42+02:00: marked 98% (159 of 162 index files)
2024-10-02T02:01:20+02:00: marked 99% (161 of 162 index files)
2024-10-02T02:01:21+02:00: marked 100% (162 of 162 index files)
2024-10-02T02:01:21+02:00: found (and marked) 141 index files outside of expected directory scheme
2024-10-02T02:01:21+02:00: Start GC phase2 (sweep unused chunks)
2024-10-02T02:02:01+02:00: processed 1% (1077 chunks)
2024-10-02T02:02:05+02:00: processed 2% (2140 chunks)
2024-10-02T02:02:09+02:00: processed 3% (3233 chunks)
2024-10-02T02:02:13+02:00: processed 4% (4319 chunks)
2024-10-02T02:02:18+02:00: processed 5% (5407 chunks)
2024-10-02T02:02:22+02:00: processed 6% (6500 chunks)
2024-10-02T02:02:26+02:00: processed 7% (7555 chunks)
2024-10-02T02:02:30+02:00: processed 8% (8678 chunks)
2024-10-02T02:02:35+02:00: processed 9% (9742 chunks)
2024-10-02T02:02:39+02:00: processed 10% (10792 chunks)
2024-10-02T02:02:43+02:00: processed 11% (11943 chunks)
2024-10-02T02:02:47+02:00: processed 12% (13045 chunks)
2024-10-02T02:02:51+02:00: processed 13% (14117 chunks)
2024-10-02T02:02:56+02:00: processed 14% (15151 chunks)
2024-10-02T02:03:00+02:00: processed 15% (16246 chunks)
2024-10-02T02:03:04+02:00: processed 16% (17330 chunks)
2024-10-02T02:03:08+02:00: processed 17% (18476 chunks)
2024-10-02T02:03:13+02:00: processed 18% (19639 chunks)
2024-10-02T02:03:17+02:00: processed 19% (20748 chunks)
2024-10-02T02:03:21+02:00: processed 20% (21800 chunks)
2024-10-02T02:03:25+02:00: processed 21% (22864 chunks)
2024-10-02T02:03:30+02:00: processed 22% (23952 chunks)
2024-10-02T02:03:34+02:00: processed 23% (25046 chunks)
2024-10-02T02:03:38+02:00: processed 24% (26125 chunks)
2024-10-02T02:03:42+02:00: processed 25% (27256 chunks)
2024-10-02T02:03:46+02:00: processed 26% (28316 chunks)
2024-10-02T02:03:50+02:00: processed 27% (29434 chunks)
2024-10-02T02:03:55+02:00: processed 28% (30599 chunks)
2024-10-02T02:03:59+02:00: processed 29% (31673 chunks)
2024-10-02T02:04:03+02:00: processed 30% (32676 chunks)
2024-10-02T02:04:07+02:00: processed 31% (33794 chunks)
2024-10-02T02:04:11+02:00: processed 32% (34876 chunks)
2024-10-02T02:04:16+02:00: processed 33% (35909 chunks)
2024-10-02T02:04:20+02:00: processed 34% (37015 chunks)
2024-10-02T02:04:24+02:00: processed 35% (38061 chunks)
2024-10-02T02:04:28+02:00: processed 36% (39177 chunks)
2024-10-02T02:04:33+02:00: processed 37% (40317 chunks)
2024-10-02T02:04:37+02:00: processed 38% (41393 chunks)
2024-10-02T02:04:41+02:00: processed 39% (42484 chunks)
2024-10-02T02:04:45+02:00: processed 40% (43527 chunks)
2024-10-02T02:04:49+02:00: processed 41% (44532 chunks)
2024-10-02T02:04:53+02:00: processed 42% (45560 chunks)
2024-10-02T02:04:57+02:00: processed 43% (46600 chunks)
2024-10-02T02:05:01+02:00: processed 44% (47699 chunks)
2024-10-02T02:05:06+02:00: processed 45% (48710 chunks)
2024-10-02T02:05:10+02:00: processed 46% (49729 chunks)
2024-10-02T02:05:14+02:00: processed 47% (50786 chunks)
2024-10-02T02:05:19+02:00: processed 48% (51867 chunks)
2024-10-02T02:05:23+02:00: processed 49% (52959 chunks)
2024-10-02T02:05:27+02:00: processed 50% (54047 chunks)
2024-10-02T02:05:32+02:00: processed 51% (55124 chunks)
2024-10-02T02:05:36+02:00: processed 52% (56164 chunks)
2024-10-02T02:05:40+02:00: processed 53% (57240 chunks)
2024-10-02T02:05:44+02:00: processed 54% (58354 chunks)
2024-10-02T02:05:49+02:00: processed 55% (59412 chunks)
2024-10-02T02:05:53+02:00: processed 56% (60551 chunks)
2024-10-02T02:05:57+02:00: processed 57% (61637 chunks)
2024-10-02T02:06:01+02:00: processed 58% (62691 chunks)
2024-10-02T02:06:05+02:00: processed 59% (63777 chunks)
2024-10-02T02:06:10+02:00: processed 60% (64806 chunks)
2024-10-02T02:06:14+02:00: processed 61% (65901 chunks)
2024-10-02T02:06:18+02:00: processed 62% (66933 chunks)
2024-10-02T02:06:22+02:00: processed 63% (67984 chunks)
2024-10-02T02:06:26+02:00: processed 64% (69048 chunks)
2024-10-02T02:06:31+02:00: processed 65% (70175 chunks)
2024-10-02T02:06:35+02:00: processed 66% (71235 chunks)
2024-10-02T02:06:39+02:00: processed 67% (72339 chunks)
2024-10-02T02:06:44+02:00: processed 68% (73444 chunks)
2024-10-02T02:06:48+02:00: processed 69% (74485 chunks)
2024-10-02T02:06:53+02:00: processed 70% (75536 chunks)
2024-10-02T02:06:57+02:00: processed 71% (76663 chunks)
2024-10-02T02:07:01+02:00: processed 72% (77741 chunks)
2024-10-02T02:07:06+02:00: processed 73% (78849 chunks)
2024-10-02T02:07:10+02:00: processed 74% (79956 chunks)
2024-10-02T02:07:15+02:00: processed 75% (81059 chunks)
2024-10-02T02:07:19+02:00: processed 76% (82120 chunks)
2024-10-02T02:07:24+02:00: processed 77% (83213 chunks)
2024-10-02T02:07:28+02:00: processed 78% (84344 chunks)
2024-10-02T02:07:33+02:00: processed 79% (85478 chunks)
2024-10-02T02:07:37+02:00: processed 80% (86589 chunks)
2024-10-02T02:07:41+02:00: processed 81% (87698 chunks)
2024-10-02T02:07:46+02:00: processed 82% (88705 chunks)
2024-10-02T02:07:50+02:00: processed 83% (89815 chunks)
2024-10-02T02:07:55+02:00: processed 84% (90861 chunks)
2024-10-02T02:07:59+02:00: processed 85% (91938 chunks)
2024-10-02T02:08:03+02:00: processed 86% (93059 chunks)
2024-10-02T02:08:08+02:00: processed 87% (94049 chunks)
2024-10-02T02:08:12+02:00: processed 88% (95152 chunks)
2024-10-02T02:08:16+02:00: processed 89% (96249 chunks)
2024-10-02T02:08:21+02:00: processed 90% (97368 chunks)
2024-10-02T02:08:25+02:00: processed 91% (98452 chunks)
2024-10-02T02:08:29+02:00: processed 92% (99513 chunks)
2024-10-02T02:08:34+02:00: processed 93% (100582 chunks)
2024-10-02T02:08:38+02:00: processed 94% (101683 chunks)
2024-10-02T02:08:42+02:00: processed 95% (102783 chunks)
2024-10-02T02:08:47+02:00: processed 96% (103902 chunks)
2024-10-02T02:08:51+02:00: processed 97% (104960 chunks)
2024-10-02T02:08:56+02:00: processed 98% (106041 chunks)
2024-10-02T02:09:00+02:00: processed 99% (107125 chunks)
2024-10-02T02:09:05+02:00: Removed garbage: 0 B
2024-10-02T02:09:05+02:00: Removed chunks: 0
2024-10-02T02:09:05+02:00: Pending removals: 814.246 MiB (in 709 chunks)
2024-10-02T02:09:05+02:00: Original data usage: 4.089 TiB
2024-10-02T02:09:05+02:00: On-Disk usage: 128.546 GiB (3.07%)
2024-10-02T02:09:05+02:00: On-Disk chunks: 107489
2024-10-02T02:09:05+02:00: Deduplication factor: 32.57
2024-10-02T02:09:05+02:00: Average chunk size: 1.225 MiB
2024-10-02T02:09:05+02:00: TASK OKBackup takes few minutes - 2-3 minutes
Is ths normal? Or how long the GC prune should take?
Thanks