[SOLVED] Deleted node still visible in web/corosync.conf

[snaz]

I removed a node from a two node cluster. I first had to set quorum_votes: 2 inside /etc/pve/corosync.conf on the remaining node.
pvecm delnode pvec1n2 worked perfectly. The removed node is offline and not connected to the network.

However the deleted node is still visible as a red entry on the web GUI. I've followed:
https://forum.proxmox.com/threads/deleted-nodes-still-visible-in-ha.87521/
https://forum.proxmox.com/threads/deleted-cluster-node-still-shows-up-in-my-gui.11614/

and deleted the folder /etc/pve/nodes/pvec1n2 on the remaining node. I also was able to restart the remaining node (and so the whole cluster). The deleted node still has an entry in the /etc/pve/corosync.conf file. If I edit the file manually and restart the cluster service, the entry of the deleted node returns and so does the web GUI entry.

root@pvec1n1:/etc/pve/nodes# pvecm status
Cluster information
-------------------
Name:             XXX
Config Version:   10
Transport:        knet
Secure auth:      on

Quorum information
------------------
Date:             Sat Jan 14 12:59:11 2023
Quorum provider:  corosync_votequorum
Nodes:            1
Node ID:          0x00000002
Ring ID:          2.401
Quorate:          Yes

Votequorum information
----------------------
Expected votes:   3
Highest expected: 3
Total votes:      2
Quorum:           2
Flags:            Quorate

Membership information
----------------------
    Nodeid      Votes Name
0x00000002          2 192.168.XX.XX (local)

 

[fweber]

Hi, are there any messages in /var/log/syslog on the remaining node?
Are you planning to only use the remaining single node from now on? If yes, you could also try to separate the remaining node from the cluster (and thus dismantle the cluster altogether) as described here.

 

[snaz]

I don't see anything strange in the syslog. Dismantling the cluster is currently no option.

 

[alexskysilk]

I removed a node from a two node cluster.

Dismantling the cluster is currently no option.

you realize that this is exactly what you are doing...

 

[fweber]

My wording was a bit unclear: By "dismantling the cluster", I meant disabling the cluster functionality on the single remaining node while keeping the installed VMs and CTs. Maybe this is what you are intending to do?

When you edited /etc/pve/corosync.conf, did you increment the config_version as mentioned here? Also, can you check whether /etc/corosync/corosync.conf and /etc/pve/corosync.conf are identical?

 

[snaz]

you realize that this is exactly what you are doing...

I think "dismantling the cluster" != "removing a node from a two node cluster". I intend to add back nodes at a later stage so I don't want to remove the cluster as a whole. There is also some storage and some backup schedules configured which if possible I don't want to reconfigure.

 

[snaz]

My wording was a bit unclear: By "dismantling the cluster", I meant disabling the cluster functionality on the single remaining node while keeping the installed VMs and CTs. Maybe this is what you are intending to do?

When you edited /etc/pve/corosync.conf, did you increment the config_version as mentioned here? Also, can you check whether /etc/corosync/corosync.conf and /etc/pve/corosync.conf are identical?

Thank you, that did the trick! I did not know that you had to increment the version.