Changing the IP address of one node in a cluster

[bea]

Hello.

I have a cluster of nodes A, B and C. I wanted to change B's IP address.

On node B I did:

systemctl stop pve-cluster
systemctl stop corosync
pmxcfs -l

Then, on node B too, I changed its IP address on
/etc/network/interfaces
/etc/hosts
/etc/pve/corosync.conf
And incremented the config version on that last file

Then reboot

Then the PVE GUI showed a red sign on node B and question marks on its VMs. I tried unsuccessfully different things to solve it.

On A and C, pvecm status does not show the new config version number of B. They were not updated and still show the old number.
On A and C pvecm nodes only shows 2 nodes: A and C.

What did I do wrong?
How could I troubleshoot it?

Thank you.

 

[waltar]

Changing an ip of a pve node should never be done ! If you need to empty the node from vm/lxc, de-join the node, reinstall the node with new ip and re-join new node to pve cluster again.

 

[bea]

I did not know.
I will go for your suggestion, it's not quick, but it seems clean and easy (I hope so
Thank you!

 

[bea]

In order to do de-join and rejoin, I am trying to follow the official Proxmox documentation but that guide does not work in the situation I am, I guess it would work with a healthy cluster.

I undid the changes explained on my first post, then I reboot the node, but still the node does not get back into the cluster.

Despite having an unhealthy cluster, I tried to follow the documentation. I powered off node B. On node A I run pvecm delnode B but I get:
Node/IP: B is not a known host of the cluster

So, I'm stack, now. The cluster does not accept node B anymore and I cannot delete node B from the cluster.

What should I do?

Thank you.

 

[UdoB]

What should I do?

Slow down.

Your Nodes A + C are still okay, right?

1)
First make a current backup of /etc on both nodes A + C. If you have a systematic backup approach --> use it. Otherwise run something like cp -av /etc /var/backups/copy-of-etc-with-damaged-node-b. Sidenote: this includes the content of /etc/pve which is a fuse mount-point of the PVE configuration database as normal files.

2)
To confirm the status of these two nodes: post the output of pvecm status of one of them.

Depending on the output of you need to clean up the situation.

Anything which belonged to B inside of /etc/pve should get deleted. Additionally check /etc/hosts on A+C and remove everything B.

3)
On B you need to follow "Separate a Node Without Reinstalling" - https://pve.proxmox.com/wiki/Cluster_Manager#_remove_a_cluster_node

Everything belonging to A or C in /etc/* should get removed.

B does contain VMs, right? Freshly joining this node to the cluster A/C is a separate task. You should find some hints here in the forum on how to do that.


Disclaimer: EVERYTHING of the above is dangerous and not "normal" - you completely operate outside the usual parameters! Also: I have never done that, so take my statements with a grain of salt...