restore a backup - failed with pilefail

C

chrischambers

Member
Nov 1, 2022
19
2
8
good evening. I hope someone can help.

I just discovered that my home assistant VM had issues so I tried to run a restore on it, and I got a error. I then try and few old ones and they all reported the same issue.
so my question is, is there a way to run a clean up on the backup file so that I can then restore it, and then get my VM back up and running.

for your reference I have a mounted share to a different server which I save my backups to.

restore vma archive: zstd -q -d -c /mnt/pve/unraid_backup/dump/vzdump-qemu-105-2024_10_15-20_49_34.vma.zst | vma extract -v -r /var/tmp/vzdumptmp444558.fifo - /var/tmp/vzdumptmp444558
CFG: size: 752 name: qemu-server.conf
DEV: dev_id=1 size: 540672 devname: drive-efidisk0
DEV: dev_id=2 size: 163208757248 devname: drive-scsi0
CTIME: Tue Oct 15 20:49:36 2024
Rounding up size to full physical extent 4.00 MiB
Logical volume "vm-105-disk-0" created.
new volume ID is 'local-lvm:vm-105-disk-0'
Logical volume "vm-105-disk-1" created.
new volume ID is 'local-lvm:vm-105-disk-1'
map 'drive-efidisk0' to '/dev/pve/vm-105-disk-0' (write zeros = 0)
map 'drive-scsi0' to '/dev/pve/vm-105-disk-1' (write zeros = 0)
progress 1% (read 1632108544 bytes, duration 4 sec)
progress 2% (read 3264217088 bytes, duration 7 sec)
progress 3% (read 4896325632 bytes, duration 11 sec)
progress 4% (read 6528434176 bytes, duration 17 sec)
progress 5% (read 8160477184 bytes, duration 22 sec)
progress 6% (read 9792585728 bytes, duration 26 sec)
progress 7% (read 11424694272 bytes, duration 29 sec)
progress 8% (read 13056802816 bytes, duration 39 sec)
progress 9% (read 14688845824 bytes, duration 43 sec)
progress 10% (read 16320954368 bytes, duration 48 sec)
progress 11% (read 17953062912 bytes, duration 57 sec)
progress 12% (read 19585171456 bytes, duration 59 sec)
progress 13% (read 21217214464 bytes, duration 62 sec)
progress 14% (read 22849323008 bytes, duration 66 sec)
progress 15% (read 24481431552 bytes, duration 75 sec)
progress 16% (read 26113540096 bytes, duration 77 sec)
progress 17% (read 27745583104 bytes, duration 82 sec)
progress 18% (read 29377691648 bytes, duration 92 sec)
progress 19% (read 31009800192 bytes, duration 96 sec)
progress 20% (read 32641908736 bytes, duration 99 sec)
progress 21% (read 34273951744 bytes, duration 103 sec)
progress 22% (read 35906060288 bytes, duration 112 sec)
progress 23% (read 37538168832 bytes, duration 114 sec)
progress 24% (read 39170277376 bytes, duration 118 sec)
progress 25% (read 40802320384 bytes, duration 129 sec)
progress 26% (read 42434428928 bytes, duration 134 sec)
progress 27% (read 44066537472 bytes, duration 137 sec)
progress 28% (read 45698646016 bytes, duration 147 sec)
progress 29% (read 47330754560 bytes, duration 151 sec)
progress 30% (read 48962797568 bytes, duration 159 sec)
progress 31% (read 50594906112 bytes, duration 165 sec)
progress 32% (read 52227014656 bytes, duration 168 sec)
progress 33% (read 53859123200 bytes, duration 178 sec)
progress 34% (read 55491166208 bytes, duration 184 sec)
progress 35% (read 57123274752 bytes, duration 189 sec)
progress 36% (read 58755383296 bytes, duration 199 sec)
progress 37% (read 60387491840 bytes, duration 205 sec)
progress 38% (read 62019534848 bytes, duration 215 sec)
progress 39% (read 63651643392 bytes, duration 222 sec)
progress 40% (read 65283751936 bytes, duration 229 sec)
progress 41% (read 66915860480 bytes, duration 237 sec)
progress 42% (read 68547903488 bytes, duration 243 sec)
progress 43% (read 70180012032 bytes, duration 246 sec)
progress 44% (read 71812120576 bytes, duration 250 sec)
progress 45% (read 73444229120 bytes, duration 258 sec)
progress 46% (read 75076272128 bytes, duration 266 sec)
progress 47% (read 76708380672 bytes, duration 273 sec)
progress 48% (read 78340489216 bytes, duration 283 sec)
progress 49% (read 79972597760 bytes, duration 291 sec)
progress 50% (read 81604640768 bytes, duration 298 sec)
progress 51% (read 83236749312 bytes, duration 305 sec)
progress 52% (read 84868857856 bytes, duration 311 sec)
progress 53% (read 86500966400 bytes, duration 317 sec)
progress 54% (read 88133074944 bytes, duration 318 sec)
progress 55% (read 89765117952 bytes, duration 319 sec)
progress 56% (read 91397226496 bytes, duration 323 sec)
progress 57% (read 93029335040 bytes, duration 326 sec)
progress 58% (read 94661443584 bytes, duration 328 sec)
progress 59% (read 96293486592 bytes, duration 332 sec)
progress 60% (read 97925595136 bytes, duration 338 sec)
progress 61% (read 99557703680 bytes, duration 345 sec)
progress 62% (read 101189812224 bytes, duration 351 sec)
progress 63% (read 102821855232 bytes, duration 354 sec)
progress 64% (read 104453963776 bytes, duration 354 sec)
progress 65% (read 106086072320 bytes, duration 357 sec)
progress 66% (read 107718180864 bytes, duration 357 sec)
progress 67% (read 109350223872 bytes, duration 357 sec)
progress 68% (read 110982332416 bytes, duration 358 sec)
progress 69% (read 112614440960 bytes, duration 359 sec)
progress 70% (read 114246549504 bytes, duration 360 sec)
progress 71% (read 115878592512 bytes, duration 363 sec)
progress 72% (read 117510701056 bytes, duration 364 sec)
progress 73% (read 119142809600 bytes, duration 364 sec)
progress 74% (read 120774918144 bytes, duration 365 sec)
progress 75% (read 122406961152 bytes, duration 365 sec)
progress 76% (read 124039069696 bytes, duration 365 sec)
progress 77% (read 125671178240 bytes, duration 365 sec)
progress 78% (read 127303286784 bytes, duration 367 sec)
progress 79% (read 128935395328 bytes, duration 369 sec)
progress 80% (read 130567438336 bytes, duration 373 sec)
progress 81% (read 132199546880 bytes, duration 379 sec)
progress 82% (read 133831655424 bytes, duration 391 sec)
progress 83% (read 135463763968 bytes, duration 394 sec)
progress 84% (read 137095806976 bytes, duration 399 sec)
progress 85% (read 138727915520 bytes, duration 410 sec)
progress 86% (read 140360024064 bytes, duration 410 sec)
progress 87% (read 141992132608 bytes, duration 410 sec)
progress 88% (read 143624175616 bytes, duration 410 sec)
progress 89% (read 145256284160 bytes, duration 410 sec)
progress 90% (read 146888392704 bytes, duration 410 sec)
progress 91% (read 148520501248 bytes, duration 410 sec)
progress 92% (read 150152544256 bytes, duration 410 sec)
progress 93% (read 151784652800 bytes, duration 410 sec)
progress 94% (read 153416761344 bytes, duration 410 sec)
_15-20_49_34.vma.zst : Decoding error (36) : Restored data doesn't match checksum
progress 95% (read 155048869888 bytes, duration 411 sec)
progress 96% (read 156680912896 bytes, duration 411 sec)
progress 97% (read 158313021440 bytes, duration 411 sec)
progress 98% (read 159945129984 bytes, duration 411 sec)
progress 99% (read 161577238528 bytes, duration 411 sec)
vma: restore failed - detected missing cluster 0 for stream drive-efidisk0
/bin/bash: line 1: 444568 Exit 1 zstd -q -d -c /mnt/pve/unraid_backup/dump/vzdump-qemu-105-2024_10_15-20_49_34.vma.zst
444569 Trace/breakpoint trap | vma extract -v -r /var/tmp/vzdumptmp444558.fifo - /var/tmp/vzdumptmp444558
unable to cleanup 'local-lvm:vm-105-disk-1' - lvremove 'pve/vm-105-disk-1' error: Logical volume pve/vm-105-disk-1 in use.
Logical volume "vm-105-disk-0" successfully removed.
temporary volume 'local-lvm:vm-105-disk-0' sucessfuly removed
no lock found trying to remove 'create' lock
error before or during data restore, some or all disks were not completely restored. VM 105 state is NOT cleaned up.
TASK ERROR: command 'set -o pipefail && zstd -q -d -c /mnt/pve/unraid_backup/dump/vzdump-qemu-105-2024_10_15-20_49_34.vma.zst | vma extract -v -r /var/tmp/vzdumptmp444558.fifo - /var/tmp/vzdumptmp444558' failed: exit code 133
 
I just ran a heath check and it looks OK

Code: 
root@pve:~# systemctl status pvedaemon pve-cluster
● pvedaemon.service - PVE API Daemon
     Loaded: loaded (/lib/systemd/system/pvedaemon.service; enabled; preset: enabled)
     Active: active (running) since Thu 2024-08-29 09:57:43 BST; 1 month 17 days ago
   Main PID: 1305 (pvedaemon)
      Tasks: 6 (limit: 38341)
     Memory: 4.2G
        CPU: 3h 7min 3.180s
     CGroup: /system.slice/pvedaemon.service
             ├─  1305 pvedaemon
             ├─443073 "pvedaemon worker"
             ├─444678 "pvedaemon worker"
             ├─445309 "pvedaemon worker"
             ├─447778 "task UPIDve:0006D522:1875FAA6:670ED6A5:vncshell::root@pam:"
             └─447779 /usr/bin/termproxy 5900 --path /nodes/pve --perm Sys.Console -- /bin/login -f root

Oct 15 21:52:15 pve pvedaemon[444678]: <root@pam> successful auth for user 'root@pam'
Oct 15 21:54:06 pve pvedaemon[447543]: starting termproxy UPIDve:0006D437:1875E53A:670ED66E:vncshell::root@pam:
Oct 15 21:54:06 pve pvedaemon[445309]: <root@pam> starting task UPIDve:0006D437:1875E53A:670ED66E:vncshell::root@pam:
Oct 15 21:54:06 pve pvedaemon[443073]: <root@pam> successful auth for user 'root@pam'
Oct 15 21:54:06 pve login[447546]: pam_unix(login:session): session opened for user root(uid=0) by root(uid=0)
Oct 15 21:55:00 pve pvedaemon[445309]: <root@pam> end task UPIDve:0006D437:1875E53A:670ED66E:vncshell::root@pam: OK
Oct 15 21:55:01 pve pvedaemon[447778]: starting termproxy UPIDve:0006D522:1875FAA6:670ED6A5:vncshell::root@pam:
Oct 15 21:55:01 pve pvedaemon[443073]: <root@pam> starting task UPIDve:0006D522:1875FAA6:670ED6A5:vncshell::root@pam:
Oct 15 21:55:01 pve pvedaemon[445309]: <root@pam> successful auth for user 'root@pam'
Oct 15 21:55:01 pve login[447781]: pam_unix(login:session): session opened for user root(uid=0) by root(uid=0)

● pve-cluster.service - The Proxmox VE cluster filesystem
     Loaded: loaded (/lib/systemd/system/pve-cluster.service; enabled; preset: enabled)
     Active: active (running) since Thu 2024-08-29 09:57:42 BST; 1 month 17 days ago
   Main PID: 1151 (pmxcfs)
      Tasks: 7 (limit: 38341)
     Memory: 64.2M
        CPU: 39min 59.235s
     CGroup: /system.slice/pve-cluster.service
             └─1151 /usr/bin/pmxcfs

Aug 29 09:57:41 pve systemd[1]: Starting pve-cluster.service - The Proxmox VE cluster filesystem...
Aug 29 09:57:41 pve pmxcfs[1137]: [main] notice: resolved node name 'pve' to '192.168.1.6' for default node IP address
Aug 29 09:57:41 pve pmxcfs[1137]: [main] notice: resolved node name 'pve' to '192.168.1.6' for default node IP address
Aug 29 09:57:42 pve systemd[1]: Started pve-cluster.service - The Proxmox VE cluster filesystem.


I also check for local storage and it is 48.77% (397.08 GB of 814.19 GB)
 
Last edited:
just before I call it a night I try one more time and here is the results - it is still failing

estore vma archive: zstd -q -d -c /mnt/pve/unraid_backup/dump/vzdump-qemu-105-2024_10_15-20_49_34.vma.zst | vma extract -v -r /var/tmp/vzdumptmp454262.fifo - /var/tmp/vzdumptmp454262
CFG: size: 752 name: qemu-server.conf
DEV: dev_id=1 size: 540672 devname: drive-efidisk0
DEV: dev_id=2 size: 163208757248 devname: drive-scsi0
CTIME: Tue Oct 15 20:49:36 2024
Rounding up size to full physical extent 4.00 MiB
Logical volume "vm-105-disk-0" created.
new volume ID is 'local-lvm:vm-105-disk-0'
WARNING: You have not turned on protection against thin pools running out of space.
WARNING: Set activation/thin_pool_autoextend_threshold below 100 to trigger automatic extension of thin pools before they get full.
Logical volume "vm-105-disk-2" created.
WARNING: Sum of all thin volume sizes (<888.02 GiB) exceeds the size of thin pool pve/data and the amount of free space in volume group (16.00 GiB).
new volume ID is 'local-lvm:vm-105-disk-2'
map 'drive-efidisk0' to '/dev/pve/vm-105-disk-0' (write zeros = 0)
map 'drive-scsi0' to '/dev/pve/vm-105-disk-2' (write zeros = 0)
progress 1% (read 1632108544 bytes, duration 4 sec)
progress 2% (read 3264217088 bytes, duration 8 sec)
progress 3% (read 4896325632 bytes, duration 11 sec)
progress 4% (read 6528434176 bytes, duration 22 sec)
progress 5% (read 8160477184 bytes, duration 24 sec)
progress 6% (read 9792585728 bytes, duration 28 sec)
progress 7% (read 11424694272 bytes, duration 40 sec)
progress 8% (read 13056802816 bytes, duration 44 sec)
progress 9% (read 14688845824 bytes, duration 48 sec)
progress 10% (read 16320954368 bytes, duration 64 sec)
progress 11% (read 17953062912 bytes, duration 67 sec)
progress 12% (read 19585171456 bytes, duration 70 sec)
progress 13% (read 21217214464 bytes, duration 72 sec)
progress 14% (read 22849323008 bytes, duration 84 sec)
progress 15% (read 24481431552 bytes, duration 92 sec)
progress 16% (read 26113540096 bytes, duration 94 sec)
progress 17% (read 27745583104 bytes, duration 106 sec)
progress 18% (read 29377691648 bytes, duration 111 sec)
progress 19% (read 31009800192 bytes, duration 114 sec)
progress 20% (read 32641908736 bytes, duration 117 sec)
progress 21% (read 34273951744 bytes, duration 126 sec)
progress 22% (read 35906060288 bytes, duration 129 sec)
progress 23% (read 37538168832 bytes, duration 131 sec)
progress 24% (read 39170277376 bytes, duration 142 sec)
progress 25% (read 40802320384 bytes, duration 148 sec)
progress 26% (read 42434428928 bytes, duration 153 sec)
progress 27% (read 44066537472 bytes, duration 159 sec)
progress 28% (read 45698646016 bytes, duration 170 sec)
progress 29% (read 47330754560 bytes, duration 175 sec)
progress 30% (read 48962797568 bytes, duration 183 sec)
progress 31% (read 50594906112 bytes, duration 190 sec)
progress 32% (read 52227014656 bytes, duration 196 sec)
progress 33% (read 53859123200 bytes, duration 210 sec)
progress 34% (read 55491166208 bytes, duration 215 sec)
progress 35% (read 57123274752 bytes, duration 220 sec)
progress 36% (read 58755383296 bytes, duration 236 sec)
progress 37% (read 60387491840 bytes, duration 243 sec)
progress 38% (read 62019534848 bytes, duration 249 sec)
progress 39% (read 63651643392 bytes, duration 264 sec)
progress 40% (read 65283751936 bytes, duration 271 sec)
progress 41% (read 66915860480 bytes, duration 277 sec)
progress 42% (read 68547903488 bytes, duration 284 sec)
progress 43% (read 70180012032 bytes, duration 290 sec)
progress 44% (read 71812120576 bytes, duration 294 sec)
progress 45% (read 73444229120 bytes, duration 301 sec)
progress 46% (read 75076272128 bytes, duration 317 sec)
progress 47% (read 76708380672 bytes, duration 324 sec)
progress 48% (read 78340489216 bytes, duration 331 sec)
progress 49% (read 79972597760 bytes, duration 346 sec)
progress 50% (read 81604640768 bytes, duration 353 sec)
progress 51% (read 83236749312 bytes, duration 364 sec)
progress 52% (read 84868857856 bytes, duration 371 sec)
progress 53% (read 86500966400 bytes, duration 381 sec)
progress 54% (read 88133074944 bytes, duration 383 sec)
progress 55% (read 89765117952 bytes, duration 383 sec)
progress 56% (read 91397226496 bytes, duration 385 sec)
progress 57% (read 93029335040 bytes, duration 387 sec)
progress 58% (read 94661443584 bytes, duration 388 sec)
progress 59% (read 96293486592 bytes, duration 392 sec)
progress 60% (read 97925595136 bytes, duration 405 sec)
progress 61% (read 99557703680 bytes, duration 408 sec)
progress 62% (read 101189812224 bytes, duration 414 sec)
progress 63% (read 102821855232 bytes, duration 421 sec)
progress 64% (read 104453963776 bytes, duration 421 sec)
progress 65% (read 106086072320 bytes, duration 423 sec)
progress 66% (read 107718180864 bytes, duration 423 sec)
progress 67% (read 109350223872 bytes, duration 424 sec)
progress 68% (read 110982332416 bytes, duration 424 sec)
progress 69% (read 112614440960 bytes, duration 424 sec)
progress 70% (read 114246549504 bytes, duration 424 sec)
progress 71% (read 115878592512 bytes, duration 425 sec)
progress 72% (read 117510701056 bytes, duration 425 sec)
progress 73% (read 119142809600 bytes, duration 425 sec)
progress 74% (read 120774918144 bytes, duration 426 sec)
progress 75% (read 122406961152 bytes, duration 426 sec)
progress 76% (read 124039069696 bytes, duration 426 sec)
progress 77% (read 125671178240 bytes, duration 426 sec)
progress 78% (read 127303286784 bytes, duration 427 sec)
progress 79% (read 128935395328 bytes, duration 430 sec)
progress 80% (read 130567438336 bytes, duration 437 sec)
progress 81% (read 132199546880 bytes, duration 449 sec)
progress 82% (read 133831655424 bytes, duration 451 sec)
progress 83% (read 135463763968 bytes, duration 454 sec)
progress 84% (read 137095806976 bytes, duration 468 sec)
progress 85% (read 138727915520 bytes, duration 470 sec)
progress 86% (read 140360024064 bytes, duration 470 sec)
progress 87% (read 141992132608 bytes, duration 470 sec)
progress 88% (read 143624175616 bytes, duration 470 sec)
progress 89% (read 145256284160 bytes, duration 470 sec)
progress 90% (read 146888392704 bytes, duration 470 sec)
progress 91% (read 148520501248 bytes, duration 470 sec)
progress 92% (read 150152544256 bytes, duration 470 sec)
progress 93% (read 151784652800 bytes, duration 470 sec)
progress 94% (read 153416761344 bytes, duration 470 sec)
_15-20_49_34.vma.zst : Decoding error (36) : Restored data doesn't match checksum
progress 95% (read 155048869888 bytes, duration 470 sec)
progress 96% (read 156680912896 bytes, duration 470 sec)
progress 97% (read 158313021440 bytes, duration 470 sec)
progress 98% (read 159945129984 bytes, duration 470 sec)
progress 99% (read 161577238528 bytes, duration 470 sec)
vma: restore failed - detected missing cluster 0 for stream drive-efidisk0
/bin/bash: line 1: 454272 Exit 1 zstd -q -d -c /mnt/pve/unraid_backup/dump/vzdump-qemu-105-2024_10_15-20_49_34.vma.zst
454273 Trace/breakpoint trap | vma extract -v -r /var/tmp/vzdumptmp454262.fifo - /var/tmp/vzdumptmp454262
Logical volume "vm-105-disk-0" successfully removed.
temporary volume 'local-lvm:vm-105-disk-0' sucessfuly removed
Logical volume "vm-105-disk-2" successfully removed.
temporary volume 'local-lvm:vm-105-disk-2' sucessfuly removed
no lock found trying to remove 'create' lock
error before or during data restore, some or all disks were not completely restored. VM 105 state is NOT cleaned up.
TASK ERROR: command 'set -o pipefail && zstd -q -d -c /mnt/pve/unraid_backup/dump/vzdump-qemu-105-2024_10_15-20_49_34.vma.zst | vma extract -v -r /var/tmp/vzdumptmp454262.fifo - /var/tmp/vzdumptmp454262' failed: exit code 133
 
You must log in or register to reply here.
Top Bottom