restore failed: Data blob has wrong CRC checksum.

R

rcarvalho

Member
Aug 1, 2022
12
0
6
Hi there.

Recently my Promox VE Server (7.2 i beleive) broke down.
I've replaced with a new server and installed the most recent version of PVE.

I'm trying to restore VMs snapshot from the Backup Server but i'm getting "restore failed: Data blob has wrong CRC checksum."

In the task viewer of PBS i'm getting success message

2023-03-28T19:24:15+01:00: download chunk "/mnt/datastore/BACKUP-VM_FTP/.chunks/784a/784aebc32939d8260c14732ccee5f70edcfec288ff89a8e080dd083560d87aeb"
2023-03-28T19:24:16+01:00: reader finished successfully
2023-03-28T19:24:16+01:00: TASK OK

However on PVE i'm getting this:

Wiping dos signature on /dev/FTP/vm-100-disk-0.
Wiping atari signature on /dev/FTP/vm-100-disk-0.
Logical volume "vm-100-disk-0" created.
new volume ID is 'FTP:vm-100-disk-0'
WARNING: device /dev/sdc is an md component, not setting device for PV.
restore proxmox backup image: /usr/bin/pbs-restore --repository rcarvalho@pbs@192.168.88.196:BACKUP-VM_FTP vm/100/2023-03-27T09:51:49Z drive-virtio0.img.fidx /dev/FTP/vm-100-disk-0 --verbose --format raw
connecting to repository 'rcarvalho@pbs@192.168.88.196:BACKUP-VM_FTP'
open block backend for target '/dev/FTP/vm-100-disk-0'
starting to restore snapshot 'vm/100/2023-03-27T09:51:49Z'
download and verify backup index
progress 1% (read 9663676416 bytes, zeroes = 0% (54525952 bytes), duration 505 sec)
progress 2% (read 19327352832 bytes, zeroes = 0% (54525952 bytes), duration 1403 sec)
progress 3% (read 28991029248 bytes, zeroes = 0% (54525952 bytes), duration 2370 sec)
progress 4% (read 38654705664 bytes, zeroes = 0% (54525952 bytes), duration 3305 sec)
progress 5% (read 48318382080 bytes, zeroes = 0% (54525952 bytes), duration 4228 sec)
progress 6% (read 57982058496 bytes, zeroes = 0% (54525952 bytes), duration 5122 sec)
progress 7% (read 67645734912 bytes, zeroes = 0% (58720256 bytes), duration 6047 sec)
progress 8% (read 77309411328 bytes, zeroes = 0% (58720256 bytes), duration 6954 sec)
progress 9% (read 86973087744 bytes, zeroes = 0% (58720256 bytes), duration 7731 sec)
progress 10% (read 96636764160 bytes, zeroes = 0% (58720256 bytes), duration 8484 sec)
progress 11% (read 106300440576 bytes, zeroes = 0% (58720256 bytes), duration 9379 sec)
progress 12% (read 115964116992 bytes, zeroes = 0% (67108864 bytes), duration 10299 sec)
restore failed: Data blob has wrong CRC checksum.
WARNING: device /dev/sdc is an md component, not setting device for PV.
Logical volume "vm-100-disk-0" successfully removed
temporary volume 'FTP:vm-100-disk-0' sucessfuly removed
error before or during data restore, some or all disks were not completely restored. VM 100 state is NOT cleaned up.
TASK ERROR: command '/usr/bin/pbs-restore --repository rcarvalho@pbs@192.168.88.196:BACKUP-VM_FTP vm/100/2023-03-27T09:51:49Z drive-virtio0.img.fidx /dev/FTP/vm-100-disk-0 --verbose --format raw' failed: exit code 255

Aparently it's not lack of disk space as the snapshot has 960GB (aprox.) and the LVM has 1TB
Captura de ecrã 2023-03-28 212328.jpg
Thank you in advance
 
Did you run a recent verify job on your PBS to see if the backup snapshot got some chunks corrupted to rule out the your backup is damaged?
 
Hi @Dunuin
Thanks for the reply.

I had not done it, but i've done it today.

Aparently it had found corrupted chunk.

Task viewer: Datastore BACKUP-VM_FTP - Verification

OutputStatus

2023-03-29T09:28:18+01:00: verify datastore BACKUP-VM_FTP
2023-03-29T09:28:18+01:00: found 1 groups
2023-03-29T09:28:18+01:00: verify group BACKUP-VM_FTP:vm/100 (2 snapshots)
2023-03-29T09:28:18+01:00: verify BACKUP-VM_FTP:vm/100/2023-03-27T09:51:49Z
2023-03-29T09:28:18+01:00: check qemu-server.conf.blob
2023-03-29T09:28:18+01:00: check drive-virtio0.img.fidx
2023-03-29T11:17:54+01:00: can't verify chunk, load failed - store 'BACKUP-VM_FTP', unable to load chunk '784aebc32939d8260c14732ccee5f70edcfec288ff89a8e080dd083560d87aeb' - Data blob has wrong CRC checksum.
2023-03-29T11:17:54+01:00: corrupted chunk renamed to "/mnt/datastore/BACKUP-VM_FTP/.chunks/784a/784aebc32939d8260c14732ccee5f70edcfec288ff89a8e080dd083560d87aeb.0.bad"

I don't know how this happens, but is there anyway to recover?
 
Last edited:
if you don't have other backup snapshots that are not corrupt, you can use proxmox-backup-debug recover index /path/to/scsi0.fidx /path/to/datastore/.chunks --ignore-corrupt-chunks --skip-crc --ignore-missing-chunks --output-path some_file.raw to recover the uncorrupted parts, but in that case I would advices to use that to recover data and re-create the guest from scratch to avoid any undiscovered problems with the restored image (corrupt or missing chunks are replaced with zero bytes!)
 
  • Like
Reactions: Dunuin
Hi there.
I need some more help please with di proxmox-backup-debug command.

I've tryed to check the documentation but i'm not getting anyware.

I've tried to run this:
proxmox-backup-debug recover index /BACKUP-VM_FTP vm/100/2023-03-27T09:51:49Z drive-virtio0.img.fidx /mnt/datastore/BACKUP-VM_FTP/.chunks --ignore-corrupt-chunks --skip-crc --ignore-missing-chunks --output-path /BACKUP-VM_FTP/output.raw

but i'm getting this:

Error: parameter verification errors

parameter 'ignore-corrupt-chunks': missing parameter value.
parameter 'ignore-missing-chunks': missing parameter value.

Usage: proxmox-backup-debug recover index <file> <chunks> [OPTIONS]
<file> <string>
Path to the index file, either .fidx or .didx.

<chunks> <string>
Path to the directorty that contains the chunks, usually
<datastore>/.chunks.

Optional parameters:

--keyfile <string>
Path to a keyfile, if the data was encrypted, a keyfile is needed
for decryption.

--skip-crc <boolean> (default=false)
Skip the crc verification, increases the restore speed by lot.

What am i doing wrong?

Thanks
 
ah, sorry, you need to explicitly set them to 'true':

Code: 
proxmox-backup-debug recover index /BACKUP-VM_FTP vm/100/2023-03-27T09:51:49Z drive-virtio0.img.fidx /mnt/datastore/BACKUP-VM_FTP/.chunks --ignore-corrupt-chunks true --skip-crc true --ignore-missing-chunks true --output-path /BACKUP-VM_FTP/output.raw
 
ok now i'm getting this:

root@PBS-SRV:~# proxmox-backup-debug recover index /BACKUP-VM_FTP vm/100/2023-03-27T09:51:49Z drive-virtio0.img.fidx /mnt/datastore/BACKUP-VM_FTP/.chunks --ignore-corrupt-chunks true --skip-crc true --ignore-missing-chunks true --output-path /BACKUP-VM_FTP/output.raw
Error: parameter verification errors

parameter 'ignore-corrupt-chunks': schema does not allow additional properties.
parameter 'ignore-missing-chunks': schema does not allow additional properties.
parameter 'output-path': schema does not allow additional properties.
 
ah, sorry, your command had a misplaced space - it should work with and without true if you fix that up:

/BACKUP-VM_FTP vm/100/2023-03-27T09:51:49Z drive-virtio0.img.fidx

vs

/BACKUP-VM_FTP vm/100/2023-03-27T09:51:49Z/drive-virtio0.img.fidx
 
hmm not yet :)

root@PBS-SRV:~# proxmox-backup-debug recover index /BACKUP-VM_FTP vm/100/2023-03-27T09:51:49Z/drive-virtio0.img.fidx /mnt/datastore/BACKUP-VM_FTP/.chunks --ignore-corrupt-chunks true --skip-crc true --ignore-missing-chunks true --output-path /BACKUP-VM_FTP/output.raw
Error: parameter verification errors

parameter 'ignore-corrupt-chunks': schema does not allow additional properties.
parameter 'ignore-missing-chunks': schema does not allow additional properties.
parameter 'output-path': schema does not allow additional properties.
 
yet another space (the first argument after index needs to be the full path to the .fidxfile)
 
root@PBS-SRV:~# proxmox-backup-debug recover index /BACKUP-VM_FTP/vm/100/2023-03-27T09:51:49Z/drive-virtio0.img.fidx /mnt/datastore/BACKUP-VM_FTP/.chunks --ignore-corrupt-chunks true --skip-crc true --ignore-missing-chunks true --output-path /BACKUP-VM_FTP/output.raw
Error: parameter verification errors

parameter 'ignore-corrupt-chunks': schema does not allow additional properties.
parameter 'ignore-missing-chunks': schema does not allow additional properties.
parameter 'output-path': schema does not allow additional properties.
 
which PBS version do you have installed? IIRC the ignore parameters were added in 2.2.2-1
 
Hi again.

With PBS upgrade, it worked.

Kown i just need to discover how can i restore the raw output image.

Thanks
 
You must log in or register to reply here.
Top Bottom