[SOLVED] Forgot to wipe old node entry in /etc/pve/nodes and re-added a new node with same name

[kfx]

Hello,

(fully updated pve 6.4 cluster)

So I decommissioned an old node "node3", I have removed it okay, the cluster is healthy.

I then configured a totally new node with the same name, same ring0 IP and added it with the GUI (join cluster and copy/pasted the info from another node) but the node was not able to fully join the cluster. I see in the logs on the new node:

[TOTEM ] Token has not been received in 8666 ms

and a "pvecm status" returns:

Cluster information
-------------------
Name:             cluster
Config Version:   20
Transport:        knet
Secure auth:      on

Quorum information
------------------
Date:             Fri Jul  9 15:20:35 2021
Quorum provider:  corosync_votequorum
Nodes:            1
Node ID:          0x00000003
Ring ID:          3.14e0
Quorate:          No

Votequorum information
----------------------
Expected votes:   5
Highest expected: 5
Total votes:      1
Quorum:           3 Activity blocked
Flags:

Membership information
----------------------
    Nodeid      Votes Name
0x00000003          1 10.10.10.3 (local)

In fear, I have powered OFF the new node.

What should I do at this point ? Can I remove the /etc/pve/nodes/node3 folder (containing, I guess, files about the previous and removed "node3" node) ?
I am in fear to broke the whole cluster if I reboot the new node3 as is

Thank you very much for any hindsight.

 

[kfx]

I add that I have rigorously followed the documentation at https://pve.proxmox.com/wiki/Cluster_Manager for removing an adding the node.
There I read:

If, for whatever reason, you want this server to join the same cluster again, you have to

    reinstall Proxmox VE on it from scratch

    then join it, as explained in the previous section.

This seems to be untrue as the removal procedure does not remove old entry in /etc/pve/nodes preventing adding another new and "same named" node.

Can I remove /etc/pve/nodes/"node3" and reboot the new server named "node3" with same IP addresses (both admin net and link0) ?

 

[kfx]

If anyone come across this thread:

The problem was 'simply' that one (and only one in a five node cluster) node wasnt seeing the new node.
I've use the 'corosync-cfgtool -s' on every node to detect it and after a 'systemctl restart pve-cluster.service corosync.service' everything is working okay !