Prune, GC and Verfy - Job Duration

[Quickly]

Hello.
The Prune, GC and Verify jobs run for a very long time.
# Now
Prune = 1x on Saturday = now runs already 3 days
Verify = 1x a month = now runs already 3 days

I would like to make my life a little easier. That's why I just write....
Will the individual jobs be faster if I just run them 1x per hour? Or once a Day?
Prune & GC?
Verify?

 

[dcsapak]

prune should definitely be faster, but gc & verify can take a while (depending on the hw) since we have to touch many (potentially small) chunks

can you post the task log of the running jobs and the syslog/journal as well?

 

[Dunuin]

Verify tasks can only be made faster by replacing all HDDs with SSDs. In case of ZFS the prune and GC tasks speed can be increased by adding a SSD as a L2ARC for metadata caching or adding SSDs as special devices so all metadata will be permanently stored on the SSDs instead of the HDDs.

 

[Quickly]

can you post the task log of the running jobs and the syslog/journal as well?

The following is a log as an example. Of the 60 jobs, only 5 are now active. Therefore one as an example.

Is it Ok?

# Dummy_NAME - Daily Backup to PBS

 

[dcsapak]

ok you have a fairly large datastore > 1million chunks and it does not help that both gc and verify run simultaniously

but as @Dunuin said, it probably boils down to getting faster hardware or some caching

 

[Quickly]

Verify tasks can only be made faster by replacing all HDDs with SSDs. In case of ZFS the prune and GC tasks speed can be increased by adding a SSD as a L2ARC for metadata caching or adding SSDs as special devices so all metadata will be permanently stored on the SSDs instead of the HDDs.

`Thank you for the info. I understand that.
The question is: without swapping disks, can you speed up the single execution by "more jobs"?
So run the jobs 4x a day, etc.

 

[dcsapak]

for gc, no, that depends always on the the number of chunks * time to access it

for verify, it's normally better to let multiple snapshots verify together, so the job can know which chunks were already verified and does not have to do it twice

i'd recommend only verifying new snaphots regularly (say daily or weekly) and reverifying older snapshots e.g. once a month