[SOLVED] Backup verification

[rian81]

Hi,

I want to ask, is the backup verification step important, because it takes a long time to do the verification.
If verification step pass, is there an effect on backup and restore?...

Thanks

 

[dcsapak]

the verification is intended to detect corruption in chunk files and missing chunks, so basically errors in the storage layer (or mishandling by an admin...)
so if you don't do that, you trust that the storage does never corrupt and the files are all written correcltly

when actually restoring backups, errors or missing files will prevent restoring, so doing it regularly makes sure everything is ok

note that some storages (e.g. zfs) already have built in checksums and with regular maintenance taks ( e.g. scrubs for zfs) that is already done on the storage layer and the gain from doing the verification in pbs is rather slim
(still can detect some issues, e.g. deleted files, etc.)

if the files are all ok, having done verification or not has no impact on restoring or backing up

 

[rian81]

noted, thanks for the explanations, clear enough

 

[christophe]

Hi,

Thank you very much for clarification.

I need some more : suppose a chunk belongs to 50 VMs. And a verification task begins (datastore or namespace level).

How many times will this chunk be verified ?

Christophe.

 

[rian81]

Based on my experience :

1. When verification scheduler run, it will sequential verification job and will take so long to do all vm verification. The CPU resource consume was rather low.
2. When you set verify new snapshot, it will paralel verification job based on new snapshot backup. But it will consume highest CPU resource (at my pbs until 95-100%)

Note : I have 4 pve cluster (16 node), 1 pbs, and about 200 VMs

 

[dcsapak]

in a single verification job, a chunk will only be verified one time

of course if you have multiple verification jobs that include the same chunks, those will be verified multiple times, but only once per job

 

[christophe]

Perfect.

Thanks.