How do I remove both nodes and recreate cluster?

[galgier]

I have two nodes: pve1 and pve2. I got both working fine and tried to create a cluster with them and I messed it up [*below]. I just want to uncluster them and recluster them correctly but they won't forget about each other.

I followed this to uncluster: https://pve.proxmox.com/pve-docs/pve-admin-guide.html#pvecm_separate_node_without_reinstall

Now when I try to add pve2 to pve1 I get:
root@pve1 3% pvecm add 192.168.11.32
Please enter superuser (root) password for '192.168.11.32': ********
detected the following error(s):
* this host already contains virtual guests
Check if node may join a cluster failed!

The web interface of both shows the other (just no green checkmark). I can see that pve2 still lists pve1 in /etc/pve/.vmlist:
root@pve2 1% cat /etc/pve/.vmlist
{
"version": 1,
"ids": {
"401": { "node": "pve1", "type": "qemu", "version": 2 },
"100": { "node": "pve1", "type": "qemu", "version": 3 },
"101": { "node": "pve1", "type": "qemu", "version": 1 },
"103": { "node": "pve1", "type": "qemu", "version": 4 }}

}
And this is not a normal file I can edit.

How can I clear this all up without reinstalling both? I would not mind redoing pve2 as it has no VMs, but I am using one of pve1's VMs.



*What I messed up: The IP addresses get stored in /etc/hosts. I changed the IPs to put the nodes on a different network. Independently, they worked fine. But when the web-based cluster setup runs it uses /etc/hosts to look up its own IP rather than consulting the IPs on the interfaces. I have since fixed both /etc/hosts files but the clustering setup on the second node still tries to refer to the first node by the bogus IP.

 

[sw-omit]

You seemed to have missed the step(s) of deleting the old nodes and deleting the configs to allow re-joining, if they both still know of eachother's existence.
They both need to not know of eachother's existence, and one of the servers needs to also not be empty and not know it has ever been in a cluster.

Create the cluster on 1 with the VM's, and you should be able to join server 2 which has no VM's to it, unless that node is still thinking it has some VM-Config somewhere.
A server with VM's can become a cluster, and a cluster with VM's can be joined with an empty server, but you can not join a cluster when the server you want to join to the cluster already has VM's