[SOLVED] Recover Deleted Windows Server VM

[racs]

I accidentally deleted a Windows Server VM today and need need need to get it back. Interesting thing is that I am currently logged into the server from this workstation (I am using it to submit this) so even though I deleted it I know it is still running and will continue to run until I log out or we lose power and the battery back up doesn't last long enough. I just backed up the user files so at least I have them. It won't be hard to set up a new server from scratch but it will eat up my weekend.

I removed it with "rm 100*" in shell. I was scrolling through the previous commands and accidentally hit enter when I was going by that one. Ugh!

Is there any way to un-delete this VM? I have read this thread (https://forum.proxmox.com/threads/recover-deleted-vm.15899/) but since my back up is so old I am certain it will not restore it. The last backup is dated 2015. I know, I should have backed up more recently but I don't know how. Someone else set this up for me. I will now have to research how to back it up so that I have a more recent image.

If I look on the local drive In the Proxmox Desktop I see a list of disk images and the one in question is still listed "vm-1000-disk-1.qcow2".

Please help this novice get this VM back.

UPDATE:
I just did some more research and found this page (https://serverfault.com/questions/521558/vm-running-perfectly-when-its-disk-image-is-deleted). I am not in a position to test this but will try tomorrow. Since the windows server is still running it may be possible to restore it. Any thought would be appreciated.

 

[Proxmox India]

I suggest you take a full backup of the server using windows server backup or Easus Todo Backup. on another VM by creating a share and pointing this server's backup on to that (\\ipaddress\sharename). this way you can make a new vm and re-install easus todo backup and restore the backup from the path where you saved the backup.

FYI Easus todo backup can backup to a fileshare, ftp, nasbox etc...

 

[udo]

I accidentally deleted a Windows Server VM today and need need need to get it back. Interesting thing is that I am currently logged into the server from this workstation (I am using it to submit this) so even though I deleted it I know it is still running and will continue to run until I log out or we lose power and the battery back up doesn't last long enough. I just backed up the user files so at least I have them. It won't be hard to set up a new server from scratch but it will eat up my weekend.

I removed it with "rm 100*" in shell. I was scrolling through the previous commands and accidentally hit enter when I was going by that one. Ugh!

Is there any way to un-delete this VM? I have read this thread (https://forum.proxmox.com/threads/recover-deleted-vm.15899/) but since my back up is so old I am certain it will not restore it. The last backup is dated 2015. I know, I should have backed up more recently but I don't know how. Someone else set this up for me. I will now have to research how to back it up so that I have a more recent image.

If I look on the local drive In the Proxmox Desktop I see a list of disk images and the one in question is still listed "vm-1000-disk-1.qcow2".

Please help this novice get this VM back.

UPDATE:
I just did some more research and found this page (https://serverfault.com/questions/521558/vm-running-perfectly-when-its-disk-image-is-deleted). I am not in a position to test this but will try tomorrow. Since the windows server is still running it may be possible to restore it. Any thought would be appreciated.

Hi,
yes - if you don't switched off the VM, you can copy the file.

Do you use lvm as storage?

I've answer a long time ago an post about this https://forum.proxmox.com/threads/deleted-image-of-running-vm.9258/#post-52518

Udo

 

[racs]

Hi,
yes - if you don't switched off the VM, you can copy the file.

Do you use lvm as storage?

I've answer a long time ago an post about this https://forum.proxmox.com/threads/deleted-image-of-running-vm.9258/#post-52518

Udo

Thank you! I am not sure if it is lvm or not.

I did what you said but I get nothing. When I hit enter the cursor just goes to the second line and blinks. I do not get a new prompt. It just sits there. I let it sit for a while and it makes not difference. I verified that both lsof and grep are installed. If I try to use sudo it says "bash: sudo: command not found". My prompt says "root@rac-vmserver:/#"

I have navigated to /var/lib/vz/images/ and 1000 is there. I assume it will disappear when I reboot the proxmox server. I have also navigated to /proc/ and there are a number of folders in the format noted in the linked posted. Not sure how to determine which folder contains the correct image.

Any advise?

 

[udo]

Thank you! I am not sure if it is lvm or not.

I did what you said but I get nothing. When I hit enter the cursor just goes to the second line and blinks. I do not get a new prompt. It just sits there. I let it sit for a while and it makes not difference. I verified that both lsof and grep are installed. If I try to use sudo it says "bash: sudo: command not found". My prompt says "root@rac-vmserver:/#"

I have navigated to /var/lib/vz/images/ and 1000 is there. I assume it will disappear when I reboot the proxmox server. I have also navigated to /proc/ and there are a number of folders in the format noted in the linked posted. Not sure how to determine which folder contains the correct image.

Any advise?

Hi,
on an heavy used host lsof can take some time, but not so much...
Are you sure, that you type the command right?

What is the output of

time lsof | wc -l
lsof | grep deleted

You don't use sudo as root (sudo is superuser do - which is root allready).
the directory /var/lib/vz/images/VMID will not disappear, if you remove the vmdisks inside, but this don't help now...

Udo

 

[racs]

Hi,
on an heavy used host lsof can take some time, but not so much...
Are you sure, that you type the command right?

What is the output of

time lsof | wc -l
lsof | grep deleted

You don't use sudo as root (sudo is superuser do - which is root allready).
the directory /var/lib/vz/images/VMID will not disappear, if you remove the vmdisks inside, but this don't help now...

Udo

Very good, thank you! I will try this when I go there tonight and let you know.

 

[racs]

Hi,
on an heavy used host lsof can take some time, but not so much...
Are you sure, that you type the command right?

What is the output of

time lsof | wc -l
lsof | grep deleted

You don't use sudo as root (sudo is superuser do - which is root allready).
the directory /var/lib/vz/images/VMID will not disappear, if you remove the vmdisks inside, but this don't help now...

Udo

I tried both and they still hung up and did not display results. I hit CTRL C to back out. I let it sit for over an hour and the shell window actually timed out on me so I could not see anything. I attached a PDF with a picture of what I typed to be sure I got it correct.


The guy who set it up for me is meeting me in the morning and is sure he can recover it. I will post what he did to get it back online.

 

[racs]

Problems Solved! Please mark this thread as solved.

I (my friend, actually) got the Windows server back and have since backed everything up. My friend who set up the server to begin with came over and corrected the issue. I am going to do my best to describe what he did and how he corrected it.

From the beginning...

I mistakenly deleted a Windows Server VM via Shell:

rm 100*

This command was in the bash history and I mistakenly hit enter as I was scrolling by it. It is an older Proxmox install so there was no warning or "are you sure?" prompt. There is now. He added it to a bash file. At the time the VM in question was running and I was logged into a workstation on the Windows Network. Since this was the case I am told, it didn't actually delete the disk image. We now know that t merely deleted the config file associated with the image which resulted in it being removed from the VM list in the Proxmox window. I also lost access to it even though I had the link to the console.

Since the file in question (VM-1000-disk-1.qcow2) was still present in "/var/lib/vz/images/1000/" I copied it to a folder on the same drive and my friend copied it to a back up drive as well just in case it disappeared when we powered the VM down. One thing to note is that we never actually rebooted the Proxmox server and don't plan to unless we have to.

My friend simply copied one of the current config files (/etc/pve/qemu-server/) and renamed it to 1000.conf and populated it with the correct details pertaining to "VM-1000-disk-1.qcow2". Since it was still present it was listed in the image list on the drive. Replacing the config file with the correct information for this VM made it appear in the VM list in Proxmox. Becuase of this we were able to go into the Windows Server and shut it down. We then proceeded to back it up to an external NFS drive. I will talk about that drive below.

This worked for us becuase the VM in question was still running and there were processes running on the server since I was logged into it via a work station. Had I deleted a VM that was not running I am told that it would have been lost for good. I plan to test this on another test VM and will report back.

At first as noted in a previous post, when I entered the following command in shell I got nothing. It seemed to just time out.

# lsof | grep 1000

What we realized is that our NFS back up system was not powered on but was enabled in Proxmox. This caused Proxmox to time out looking for this NFS drive. Once we powered it on and entered the above command we got the details about the VM. But those details did not indicate that the image was deleted. As mentioned, it was still in "/var/lib/vz/images/1000/".

When we used the following command we saw lots of files that were deleted and the files were truly gone.

lsof | grep deleted

I hope this helps someone.