[SOLVED] Cluster, LXC, VM are all gone on 5 nodes

proxymax

Member
Jun 21, 2023
8
1
8
So I accidentally ran a remove command on the root directory of a node, which took the node offline and then I noticed I couldn’t login to any of the other nodes. I rebooted everything and found out the cluster was gone along with all the VM and LXC.

There is nothing in /etc/pve/ on any of the nodes. I just have config.db but not sure how I can rebuild. How can I recover atleast one of my VM with has 70 TB of data? I read that the data is still there and just need to rebuild.

I have read throughout online but did not find anything so my last hope is here. I appreciate all your help. I am devastated and have been working on this for quite sometime and I need to get up and running quickly.
 
Last edited:
Wow, that is very unfortunate. Recovery is described here.

PS: It is important to also have a backup of your nodes itself or at least the /etc/pve directory.
Thanks that’s a good start though the link isn’t detailed on what to do with the config.db. Not sure what this is talking about and what to do, “Following this, adapt /etc/hostname and /etc/hosts according to the lost Proxmox VE host, then reboot and check (and don’t forget your VM/CT data).”