Rebuilding a node's /etc/pve

[thewball]

Hello!

I messed up and removed a node from my cluster. I then brilliantly managed to delete /etc/pve. I was able to scp a copy from another node, but I was wondering if anyone knows what I would need to change to mirror the old node's settings on the broken node. I would like to readd the node back to the cluster once I get this nightmare figured out. I am running ZFS without ceph.

Do I have to transfer the contents of /nodes/nodename to /local?

I have some running vm's on this machine I would hate to reboot and have them fail, so I am trying to do as much as I can to avoid a reboot here. Is there any way I can save the currently running qemu machines in case something goes horribly wrong again?

Thanks for the help!

 

[fschauer]

On PVE /etc/pve is a fuse mount, which can be mounted with pmxcfs.

 

[fschauer]

When using LVM, the VM disks can be found at /dev/pve.
When using disk images, these can be found at /var/lib/vz/images.

 

[thewball]

After realizing I needed to copy /var/lib/pve-cluster/config.db instead of /etc/pve, I am able to see all nodes in the cluster from the web view on the problematic node, but all other nodes are showing offline on the problematic node. From the currently working cluster, I can see the problematic node with no status icon since it was removed from the cluster. I confirmed I can ping all of the other nodes and have access to the machines.

Any advice on how to re add this node to the cluster?

Thank you!!

EDIT Current pvecm status:

Cluster information
-------------------
Name:             PROD
Config Version:   10
Transport:        knet
Secure auth:      on

Cannot initialize CMAP service

Edit 2:
Navigating to any host that is offline or with no status on the working cluster and problematic node results in the error:
hostname lookup 'HOSTNAME' failed - failed to get address info for: HOSTNAME: Name or service not known (500)

 

[fschauer]

To re-add a node to a cluster, you can try running the following command from the node you want to join into the cluster:

pvecm add <hostname> --force