Datastore Verify on verified snapshots

[vikozo]

Hallo

on my PBS I have set the verification to every hour.
after a long run all snapshots are verified.
Why to rerun the verification it still need a lots of time, because there have been no update added since.
if the file is tagged as verified you could just skip an go to the next file until you go a file without tag and to verifiy.

just asking because this task is stressing and heating my disk much more the the cc, prune and the backups

have a nice day
vinc

 

[tafkaz]

Hi,
yes, wondering here too.
Why reverify every backup that has allready been verified?
Would be good to verify just the new arrivals, don't you think?
This way it would be much easier to run a verification at least every day or so.
thanks
Sascha

 

[Cookiefamily]

Hi,
To understand why this is not done on every Backup and why it verifies old backups again you have to understand what „Verify“ does:

It calculates hashes of all chunks and compares them to what they were when getting backed up first. This helps to find bits that flipped or are rotten just due to the physical nature of magnetic drives.
This process is very I/O intensive. And doing this on every backup is not really a feasible idea, let’s say you get backups of your Linux VM at 0:00, 0:05, 0:10 etc... the Linux VMs probably share the same base files that will get deduplicated. This means you could be verifying the exact same chunks (because deduplication) every 5 minutes.

this does not really create a benefit and really hurts your I/O. I‘d rather set up a cron to verify each week.

 

[tafkaz]

Hi @Cookiefamily, thanks for that great explanation.
Once a week should be fine then.
Cheers
Sascha

 

[fabian]

note that improvements for selecting which snapshots to verify are on the way - e.g., last X, only unverified, or older than XX days. you can then setup e.g. daily runs for verifying new incoming snapshots, and weekly or monthly to keep the verification status 'fresh' without taxing your disks too much.

 

[tafkaz]

Oh that is in fact incredible news!
Thanks for your very very great work, highly appreciated!

 

[vikozo]

@fabian danke for this good news
@Cookiefamily thanks to tell how it works to better understand.

 

[tom]

Available with latest updates, really useful!

 

[vikozo]

Hello @tom
yes it is nice! thanks a lot!

"feature request"
if you setup the verifivation Job and the task is running it shows up in "Running Tasks"

would be nice to have the information like
verificationjob - "datastore Name"

have a nice day
vinc