Urgent - VM deleted but config still here

[Elmani335]

Hey every ! so here is my issue,

I was doing some networking and had to shutdown my servers, but when I pluged back everything on (like always I have autorestart 1 day per week)

But then.. my windows server wasn't booting up.. this is what I get :


I think the VM image got deleted but I don't know how, and I tryed some recover stuff but no service run with this VM-Id or anything else

I builded this vm in the next way :



(HD means = Hard Disk)
the HD(ide 0) is the boot drive
the HD(ide 1) is where is my stored data (the one that I need to recover ! )

And this is a shell output gives me a little hope :


I created another Windows server, completely blank VM, Is there any way to attach the HD ide1 (15 000G) to the new windows and try recover data ?


Or this here any other way of recovering the .qcow2 to restore the VM ?

I don't have any proxmox backups for it, I really need to recover the data it is really important.

Hope someone have a solution for this issue

=====[EDIT 1]=====
Succesfully recovered all the REALLY importand data ! (Attached the big disk to another windows server VM worked)

But know if there is any way to recover the data of the .qcow2 (ide0 100G) I would apreciate ! [it contains my website + other certfiles .. ]

If not possible, I can still reconfigure all but will literally take me an entire week
================

 

[fabian]

what kind of storage is RAID_DATA? could you post the output of "pvesm status" and "pvesm list RAID_DATA", as well as the contents of /etc/pve/storage.cfg ?

 

[Elmani335]

what kind of storage is RAID_DATA? could you post the output of "pvesm status" and "pvesm list RAID_DATA", as well as the contents of /etc/pve/storage.cfg ?

hey ! thx for the reply, my RAID_DATA is a raid array of 6 SAS hard drives powered by a H710 RAID card, here are the screen of the different outputs :

PVE version : 7.3-3

the actual disk on my PVE :

pvesm status


pvesm list RAID_DATA


storage.cfg



Hope this can help to debug the issue.

 

[fabian]

looks like the volume was (accidentally?) deleted at some point since the last successful start of the VM.. note that it could have happened while the VM was running - the file will only actually be gone once the last process that has it open exits..

doesn't hurt to also double check that it's not just in an unexpected location, e.g. by doing

find /mnt/pve/RAID_DATA -type f -iname ".*320.*"

but I am afraid you will have to recover the contents from memory (and I'd highly recommend setting up backups to avoid this situation in the future!)

 

[Elmani335]

looks like the volume was (accidentally?) deleted at some point since the last successful start of the VM.. note that it could have happened while the VM was running - the file will only actually be gone once the last process that has it open exits..

doesn't hurt to also double check that it's not just in an unexpected location, e.g. by doing

find /mnt/pve/RAID_DATA -type f -iname ".*320.*"

but I am afraid you will have to recover the contents from memory (and I'd highly recommend setting up backups to avoid this situation in the future!)

Yup, I configured monthly entire backuops for all the important VMs,

here is the find command output (nothing) :
[IMAGE 1]


also, before all of that I had this issue, I booted server, and none of the servers were starting :
[IMAGE 2]


The error was a XFS corruption (cannot fit the whole output of the coruption in the image) :
[IMAGE 3]


And to fix it, it was saying me to do "xfs_repair /dev/sda1" (we can see it on the red lines)
wich I did (/dev/sda1 is the RAID_DATA)

And here the final output after "succesful" reparation :
[IMAGE 4] <<< important



And what actually happened, is that the RAID_DATA disk identifier changed after the reboot ??
before identifier :
[IMAGE 5]


after identifier :
[IMAGE 6]


Actaully the first one isn't existing anymore, or at least I can't find it.

You will probably tell me that there is no chance to recover it, but the "320.conf" file is still there :
[IMAGE 7]



I didn't restart the server between the xfs_repair and now.

And I know that XFS is a block file system, so I hope some data recover possibility maybe ?

And here an important line present on IMAGE 4


(sorry for the long text, I give all the info I have in my hands to figure out a way to possibly recover all of that)

 

[fabian]

yeah, that sounds like xfs_repair detected corruption surrounding your missing image file and removed it.. I am not a data recovery expert, but the usual approach in such a situation would be to do a bit-wise copy of the disks before attempting such a repair/recovery, in order to try multiple possible destructive techniques with as much of the data around as possible. if the data is important to you, I would suggest not doing any more changes and contacting a company specializing in such recovery.

 

[Elmani335]

yeah, that sounds like xfs_repair detected corruption surrounding your missing image file and removed it.. I am not a data recovery expert, but the usual approach in such a situation would be to do a bit-wise copy of the disks before attempting such a repair/recovery, in order to try multiple possible destructive techniques with as much of the data around as possible. if the data is important to you, I would suggest not doing any more changes and contacting a company specializing in such recovery.

Yup, thx ! I will try to do tests because it's in a RAID array, I should probably have some more info on my RAID controler dashboard, will try some XFS recovering tool

Also I can't figure out how to install and use the xfs_undelete tool wich is on github, there isn't any real instructions on how to ? and since I haven't turned off the server still, there should be some data laying arround on the drive/ram/temp junk file