Lost all VMs. Any way to recover ?

matamata4

New Member
Aug 8, 2014
7
0
1
Hello dear Proxmox comunity,

My name is Maresand togheter with a friend of mine we have opened a small hosting service using PROXMOX and OVH.

Everything ran smoothly untill 4 hours ago when somehow, someone hacked into one of our proxmox ve accounts and deleted all the vms (3 nodes with aprox 100VMs)

I would want to know if there is any way to recover the data of the vms and then recreate them based on the data we recover.

This would be a huge life saver and whoever can help us would be paid back.

I would want to note that most of our vms ran freebsd and they were made using virtio and qcow.

Please help us recover the data and make our customers happy again.
 
Last edited:
Hello Mares,

Terrible incident. The recovery might be s challenge.
I can only give you a brief idea. Nothing for action. Data recovery is not straight forward but if you lucky enough you can get your data back.

You need to split work on two jobs: investigate from where intrusion come from and restore data. Security needs to be fixed before going live as it might happen again. Because you not sure where it comes from - the easiest way will be to rebuild cluster from scratch on spare or brand new hardware and important to use brand new passwords. Enough to have just one node just to start from.

I presume your backups gone as well and you have no remote offsite backups do you?
If you have backups - just restore. Unlikely kvm has vulnerability to escalate privileges from vm to hypervisor level.

What type of storage you are using? local, shared, SAN?
 
Hello Mares,

Terrible incident. The recovery might be s challenge.
I can only give you a brief idea. Nothing for action. Data recovery is not straight forward but if you lucky enough you can get your data back.

You need to split work on two jobs: investigate from where intrusion come from and restore data. Security needs to be fixed before going live as it might happen again. Because you not sure where it comes from - the easiest way will be to rebuild cluster from scratch on spare or brand new hardware and important to use brand new passwords. Enough to have just one node just to start from.

I presume your backups gone as well and you have no remote offsite backups do you?
If you have backups - just restore. Unlikely kvm has vulnerability to escalate privileges from vm to hypervisor level.

What type of storage you are using? local, shared, SAN?

Thank you for the reply,

We were using local storage only.
 
I assume you lost all disk images, of course, and you're trying to recover them. Any tool I know is listed above.
If nothing works, you could try to contact some specialized data recovery company, like ontrack, but it'll cost you much, and there's no guarantee. If, by any chance, any vm is still running, even if vm image is lost, there is still a chance to recover the disk: see this thread http://forum.proxmox.com/threads/9258-Deleted-image-of-running-vm

Marco
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!