I wanted to provide this resolution to an earlier thread on the topic, but was unable to locate it - hence the new thread.
For a while now I have been battling an issue of a 1TB VM backup failing verification repeatedly. This issue appeared only with this VM (out of ~20) across many different storage mediums (all backups of it reset on each). The only constants were that particular VM, that all others were no more than 500 gig, and that the verification failure always stemmed from a couple of chunks not loading.
Backups always ran flawlessly, atime was always set, storage media showed no corruption. Because of this issue, I have had the PBS install lug around an old data store with the only successfully verified backup of that VM - so I at least had that data in case of disaster. Worryingly enough this was kept on storage with one of the mirrored disks starting to flake out.
I then recently read yet another thread from someone experiencing this issue, which near the end had a staff recommendation of trying out PBS 3 (this server was still running 2.4). Somewhat desperate I gave that a go yesterday and started a backup & verify cycle.
It worked! I now have a verified backup of that VM as well on fresh storage media.
My best guess is I somehow was unlucky enough to have hit one of the zfs bugs which had a fix included in the version of Debian PBS 3 comes with. I wanted to leave this post here for anyone who might be similarly trapped. This has been a very long and not very entertaining journey.
For a while now I have been battling an issue of a 1TB VM backup failing verification repeatedly. This issue appeared only with this VM (out of ~20) across many different storage mediums (all backups of it reset on each). The only constants were that particular VM, that all others were no more than 500 gig, and that the verification failure always stemmed from a couple of chunks not loading.
Backups always ran flawlessly, atime was always set, storage media showed no corruption. Because of this issue, I have had the PBS install lug around an old data store with the only successfully verified backup of that VM - so I at least had that data in case of disaster. Worryingly enough this was kept on storage with one of the mirrored disks starting to flake out.
I then recently read yet another thread from someone experiencing this issue, which near the end had a staff recommendation of trying out PBS 3 (this server was still running 2.4). Somewhat desperate I gave that a go yesterday and started a backup & verify cycle.
It worked! I now have a verified backup of that VM as well on fresh storage media.
My best guess is I somehow was unlucky enough to have hit one of the zfs bugs which had a fix included in the version of Debian PBS 3 comes with. I wanted to leave this post here for anyone who might be similarly trapped. This has been a very long and not very entertaining journey.