HELP - boot failed: not a bootable disk after failed backup

vmunich

New Member
Sep 4, 2013
5
0
1
Ok, this is a .qcow2 Windows 2012 VM with VirtIO drivers. I have a very important database running in it, if I could only recover that DB it would be great.

The VM was running fine but then I decided to transfer it to another node. This other node is not Proxmox based so I couldn't use the new Proxmox backup tool (which generates .vma files) so I used this command to make a backup:

dd if=/var/lib/vz/images/402/vm-402-disk-1.raw | gzip | dd of=/var/lib/vz/images/402/vm-402-disk-1.gz bs=4096

I forgot to shutdown the VM before I executed the command above, so I stopped the dd process. Then I decided to forget about DD and just use the Proxmox's backup tool instead because I found a tutorial teaching how to convert the .vma backups to .tgz. Since we can set the backup mode to "Stop", I didn't shut the VM down, and asked the backup tool to do it for me.

The backup tool couldn't shutdown the VM by itself, so the backup failed.

This is the log that Proxmox generated before my VM got screwed..:

Sep 04 00:38:25 INFO: Starting Backup of VM 402 (qemu)
Sep 04 00:38:25 INFO: status = running
Sep 04 00:38:28 INFO: backup mode: stop
Sep 04 00:38:28 INFO: bandwidth limit: 60000 KB/s
Sep 04 00:38:28 INFO: ionice priority: 7
Sep 04 00:38:28 INFO: stopping vm
Sep 04 00:48:29 INFO: VM quit/powerdown failed - got timeout
Sep 04 00:48:29 ERROR: Backup of VM 402 failed - command 'qm shutdown 402 --skiplock --keepActive --timeout 600' failed: exit code 255

Then I opened the console of that VM and the VM was trying to boot but showing this message: boot failed: not a bootable disk after failed backup
I restarted the VM several times, using write back, write back (unsafe), write through, no cache, and nothing works.

I tried to mount the .qcow2 image but I failed. I converted the .qcow2 to .raw, downloaded ntfs-3g so debian could recognize the ntfs partitions and tried to mount the .raw image and it failed too. I tried running both Windows 2012 and Windows 7 isos and tried to repair the Windows installation but both failed too.

Is there any way I could at least access the files inside the image??? I had a firebird database running in it, I just need to recover a 3gb .FDB file so I can restore it in a new VM.

Help

Thanks.

Edit: I loaded the Windows 7 ISO and it shows that my drive has 120gb of free space in 120gb of unallocated space.. does it mean that I lost everything ???????? :/

Linux disk utility shows this: http://d.pr/i/pZuF

Is there anything I can do to try to recover the files??
 
Last edited:

symmcom

Well-Known Member
Oct 28, 2012
1,077
26
48
Calgary, Canada
www.symmcom.com
O man, soo sorry to hear what happened to your Windows :(

I can feel your pain as i faced the similar issue once. But in my case after trying for 24 hours without any result i just restored the VM from a 3 days old backup. Although there was some data loss, but not a huge one since 2 of the 3 days were weekend.

At later time i tried to recreate the same issue just to see what happens and i was able to recover about 94% of data by following solution:
!!! MUST COPY THE DAMAGED VIRTUAL DISK SOMEWHERE TO PREVENT FURTHER WRITE ON IT. USE IT AS MASTER COPY !!! THE MORE YOU WILL TRY TO DO SOMETHING ON THE MASTER COPY IT WILL REDUCE THE CHANCE OF RESTORE !!!

1. Create a fresh Windows VM.
2. Add the damaged Virtual Disk as 2nd disk.
3. Install program like Active Undelete.
4. Create a new partition on the 2nd damaged virtual disk.
5. Use Active Undelete to restore deleted/damaged partition/files.

Let us know how this works out for you. Good Luck!!
 

vmunich

New Member
Sep 4, 2013
5
0
1
O man, soo sorry to hear what happened to your Windows :(

I can feel your pain as i faced the similar issue once. But in my case after trying for 24 hours without any result i just restored the VM from a 3 days old backup. Although there was some data loss, but not a huge one since 2 of the 3 days were weekend.

At later time i tried to recreate the same issue just to see what happens and i was able to recover about 94% of data by following solution:
!!! MUST COPY THE DAMAGED VIRTUAL DISK SOMEWHERE TO PREVENT FURTHER WRITE ON IT. USE IT AS MASTER COPY !!! THE MORE YOU WILL TRY TO DO SOMETHING ON THE MASTER COPY IT WILL REDUCE THE CHANCE OF RESTORE !!!

1. Create a fresh Windows VM.
2. Add the damaged Virtual Disk as 2nd disk.
3. Install program like Active Undelete.
4. Create a new partition on the 2nd damaged virtual disk.
5. Use Active Undelete to restore deleted/damaged partition/files.

Let us know how this works out for you. Good Luck!!
Oh my God thank you for giving me some hope man! I was already hopeless, tried testdisk but it didn't work, Im trying photorec now but it takes longer so while photorec tries to find anything I will try what you suggested me right now and see how it goes!!!
 

vmunich

New Member
Sep 4, 2013
5
0
1
Oh my God thank you for giving me some hope man! I was already hopeless, tried testdisk but it didn't work, Im trying photorec now but it takes longer so while photorec tries to find anything I will try what you suggested me right now and see how it goes!!!
It didn't work.. I tried sereval softwares, testdisk, photorec, loaded the damaged image on another windows VM and ran a bunch of softwares but they managed to recover only ~8gb..

This is the log that Proxmox generated before my VM got screwed..:

Sep 04 00:38:25 INFO: Starting Backup of VM 402 (qemu)
Sep 04 00:38:25 INFO: status = running
Sep 04 00:38:28 INFO: backup mode: stop
Sep 04 00:38:28 INFO: bandwidth limit: 60000 KB/s
Sep 04 00:38:28 INFO: ionice priority: 7
Sep 04 00:38:28 INFO: stopping vm
Sep 04 00:48:29 INFO: VM quit/powerdown failed - got timeout
Sep 04 00:48:29 ERROR: Backup of VM 402 failed - command 'qm shutdown 402 --skiplock --keepActive --timeout 600' failed: exit code 255


Can't you guys just leave the old backup tool (< Proxmox 2.2 )for the ones that would rather use it ?????????

It would have been so much smooth if it was still available (for me) :(

Sep 04 00:48:29 INFO: VM quit/powerdown failed - got timeout

I noticed that more people got that same error while doing a backups.. But Proxmox's team insist that their .vma backups are much better and stable than the old way.. I'm not sure. Ins't Proxmox as reliable as it used to be before? :(

OK I know that maybe the problem was that Proxmox wasn't able to shut the Windows VM down, but.. I still miss the old backup tool..
 
Last edited:

symmcom

Well-Known Member
Oct 28, 2012
1,077
26
48
Calgary, Canada
www.symmcom.com
Didyou try file restore or the entire partition restore? It must be volume/partition restore.

Did you have cache enabled on the VM when everything was working? When cache enabled and vms are abruptly shut down, it causes unrecoverable error specially for windows. Same as raid cache that's why raid controller have battery backup. You are probably already familiar with this.

Proxmox staff can correct me if I am wrong, but the new backup seems to be little faster and far stable, at least for all my dozen clusters.

Sent from my SGH-T989D using Tapatalk 2
 

e100

Active Member
Nov 6, 2010
1,235
24
38
Columbus, Ohio
ulbuilder.wordpress.com
Thats what I was thinking too, maybe writeback was enabled.
All of my disks are set to cache=none, we do not leave it at default, actually select cache=none just in case the default were to ever change.

The other problem is trying to automatically shutdown windows, very unreliable in my experience. Without a clean shutdown problems will happen.
I find that most of my Windows machines require two shutdown commands to actually shutdown, first one always fails, 2nd works 90% of the time.

I prefer snapshot backups myself.
I have only ever had one snapshot backup, fail to restore, but another backup of same VM did restore fine.
We backup each VM 3X weekly and do test restores of the lastest backup of every VM each quarter, we have easily done over 800 snapshot backup restores with just one failure.
No 2012 VMs yet, only 2003 and 2008
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!