keep last X vs hourly vs xxx

SteveITS said:
I had to wrap my head around it a bit, too. All backups in PVE are full backups. With PBS only the "new chunks" are saved to disk since the existing ones already exist. So the backup of a 60 GB disk is shown as 60 GB even if most of it is zeroes. That's the deduplication at work. Our dedupe factor I think was 110ish until we added several VMs last week so now it's around 70.

2 hours for GC seems like a lot. On our (admittedly brand new) server it takes under a minute. Do you have the new GC cache set high enough? See this thread for instance.

In the task log for the backup, on the server/node, you should see that:
Code: 
...
INFO: started backup task '24e52e79-1e76-49f4-8b2d-f1765e55c3b8'
INFO: resuming VM again
INFO: scsi0: dirty-bitmap status: OK (9.3 GiB of 400.0 GiB dirty)
INFO: using fast incremental mode (dirty-bitmap), 9.3 GiB dirty of 400.0 GiB total
INFO:  16% (1.5 GiB of 9.3 GiB) in 3s, read: 517.3 MiB/s, write: 488.0 MiB/s
INFO:  35% (3.4 GiB of 9.3 GiB) in 6s, read: 629.3 MiB/s, write: 514.7 MiB/s
INFO:  54% (5.1 GiB of 9.3 GiB) in 9s, read: 590.7 MiB/s, write: 582.7 MiB/s
INFO:  76% (7.2 GiB of 9.3 GiB) in 12s, read: 706.7 MiB/s, write: 508.0 MiB/s
INFO:  89% (8.3 GiB of 9.3 GiB) in 15s, read: 401.3 MiB/s, write: 388.0 MiB/s
INFO: 100% (9.3 GiB of 9.3 GiB) in 18s, read: 340.0 MiB/s, write: 312.0 MiB/s
INFO: Waiting for server to finish backup validation...
INFO: backup is sparse: 192.00 MiB (2%) total zero data
INFO: backup was done incrementally, reused 391.82 GiB (97%)
INFO: transferred 9.33 GiB in 21 seconds (455.0 MiB/s)
INFO: adding notes to backup
INFO: Finished Backup of VM 112 (00:00:21)
...
Click to expand...
I haven't adjusted the gc cache as of yet but might consider it after/whenever I get my space back. I originally had the gc to run once a week (remember something like I needed to have my files there for 24 hours before the gc could determine dupes - from somewhere) but have not changed it down to 2xday in hopes of quickly reclaiming my space. My theory in this is that for those that are already or have been there for a while - alot of those will already be accounted for when my gc runs - and therefore I can get back on top of the storage constraints I'm facing now.
BTW - Still hasn't removed that t.5Tb as of yet.
 
I was just thinking it might speed up the GC. On our local (off site) server that doesn't use SSD the GC task also completes in well under a minute, though it has much less data than yours. I think it was in the thread I linked, that it seems to really help some people.
 
An update for everyone -
I'd say their clock is off but within 48 hours, my 'clean up' of old data happened and I was able to remove about 1.5Tb of space. Not sure why it took so long but storage is back under control now.
A follow up question:
If I have, say 4.8Tb of local data and am synching, via pbr the same amt of data then my remote data should be the same. Obsiviously if my retention is higher, ie - yearly or additional monthly specified at the remote/dr side - then that should increase. Is this what others are finding as well?

Also - what or how are you accounting for the "1" in the 3-2-1 metho
 
You must log in or register to reply here.
Top Bottom