/etc/pve/nodes accidentally deleted

Jul 3, 2019

The title says it all, I've mistakenly rm -R /etc/pve/nodes on a quorated PVE 5.4 2-node cluster.

The cluster still runs happily, and pvecm status looks fine and all. But corosync did its job thoroughly removing this directory from the two nodes. I guess the configuration is still in RAM and I'm afraid my cluster won't survive a reboot or something.

Have any of you experienced this, and know how to get out of this situation?

There is nothing critical on this cluster, it's just being deployed and tested to be put in production in the next few weeks. However, it's a reminder of how easy it is to break things, and I believe learning how to recover from this silly mistake now will be very valuable.

Also, what is the appropriate strategy to backup/restore the corosync configuration? I haven't found anything about this in the docs.

In the nodes folder there are, amongst others, all CT/VM configs stored. Best to have a backup of the /etc/pve (eg. rsync).
├── node1
│   ├── lrm_status
│   ├── lxc
│   ├── openvz
│   ├── priv
│   ├── pve-ssl.key
│   ├── pve-ssl.pem
│   └── qemu-server
├── node2
│   ├── lrm_status
│   ├── lxc
│   ├── openvz
│   ├── priv
│   ├── pve-ssl.key
│   ├── pve-ssl.pem
│   └── qemu-server
└── node3
    ├── lrm_status
    ├── lxc
    ├── openvz
    ├── priv
    ├── pve-ssl.key
    ├── pve-ssl.pem
    └── qemu-server
Corosync itself has the config under '/etc/corosync' and any change made to /etc/pve/corosync.conf, copies the file to /etc/corosync/ on all nodes. But the corosync config should still be in /etc/pve/corosync.conf.
I just made the same f@ckup. I managed to restore the configuration by:
1. Without restarting anything copy the /var/lib/pve-cluster/config.db
2. Stop pve-cluster and corosync services
3. Restore the /var/lib/pve-clusterconfig.db
4. Restart the services

Anyway backup of the configuration files is on the top of my tasks list now :)
  • Like
Reactions: kzaoaai


The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!