[SOLVED] Accidentally deleted /etc/ in one node

A

ahmed.samer

New Member
Feb 9, 2021
3
2
3
38
Hello everyone,
I was planning to remove one node from the cluster, and to do so I need to shut down this node and make sure that it will never come back up with the same configurations.

I managed to delete /boot/, but instead of deleting /etc/pve/corosync.conf I deleted /etc/
and this deleted the content of /etc/pve/ from the whole cluster because of the pmxcfs, Checked If i have any backup on /var/lib/pve-cluster/backup/ but couldn't find any.

Since everything is deleted from the /etc/pve
  • The pveproxy complained about the certificates
    Fixed by pvecm updatecerts --force

  • Cluster out of sync
    I still have the /etc/corosync/corosync.conf

  • No VM configured on any of my nodes
    I managed to re-create the configurations of the VMs on each node

  • pvecm status and pvecm nodes showing that everything is okay with one warning
    Corosync config '/etc/pve/corosync.conf' does not exist - is this node part of a cluster?

  • No storage
    Manually created /etc/pve/storage.cfg

In the end, My question will be.

Is there any way to assemble this cluster back together?

Regards
 
Last edited:
Hello everyone,
May I ask what is the optimal solution to backup the cluster configuration to avoid situations like this in the future?

Regards
 
Hi,

there is a script in GitHub but we don't support it [0]

However, if you are using Proxmox Backup Server you could create a hook-script and put it in your cron job to make a backup every week example, for example, backup for /etc/ directory [1]

Bash: 
proxmox-backup-client backup etc.pxar:/etc --repository 192.168.1.3:PBS

NOTE: make sure to double check your backup and don't blindly restore the backup of /etc, but specifically, restore individual config files or use them as reference.

[0] https://gist.github.com/mrpeardotnet/6bdc4b504f43ce57fa7eaee96d376edf
[1] https://pbs.proxmox.com/docs/backup-client.html
 
Last edited:
You must log in or register to reply here.
Top Bottom