[SOLVED] Remove node from cluster

mira said:
Check if there are still leftover files in /etc/pve/nodes/<deleted-node>.
As long as there are files still in there, the node will show up in the UI.
Please make a backup of the files before you delete them, just to be safe!
Click to expand...
yes, I can see the folder.../etc/pve/nodes/<deleted-node>
I took a backup and deleted that folder...the node has disappeared.
thanks very much
 
fahadshery said:
yes, I can see the folder.../etc/pve/nodes/<deleted-node>
I took a backup and deleted that folder...the node has disappeared.
thanks very much
Click to expand...
How do you delete this folder?
I keep getting told that my root user is not allowed to do anything in /etc/pve directory.
 
I just went through some self inflicted pain when I added a node to my cluster when one of my other nodes was offline. I didn't know that would cause so many problems. A warning in the UI would have been nice. The offline node when brought back online just didn't want to behave correctly. It hung in the shell about 30 seconds after boot and never attached to the cluster again. I could only SSH to it. I couldn't find a solution to fix its membership in the cluster. At this point it appeared best to remove it from the cluster. This was all on 8.2.x

From a good cluster node:

Code: 
pvecm nodes
pvecm delnode<nodename>

The node will still show as a ghost in the Datacenter view until the node is deleted from at least one nodes /etc/pve/nodes dir.
As long as there are files still in there, the node will show up in the UI.

Code: 
rm -r /etc/pve/nodes/<nodename>

On removed node to make it standalone and files editable:

Code: 
systemctl stop pve-cluster
/usr/bin/pmxcfs -l

rm -f /etc/pve/corosync.conf
rm -f /etc/corosync/corosync.conf
rm -f /etc/corosync/authkey
cd /etc/pve/nodes
rm -r <othernodename>  #for all other nodes

reboot now

Node is now standalone. The node can now be re-added to a cluster if desired.
 
  • Like
Reactions: luckyluk83
Thank you!
I was able to run:
Code: 
pvecm rm {nodename}
and then edit /etc/pve/ceph.conf and remove any reference to the IP of the 'unknown' node from /etc/pve/ceph.conf and then it worked flawlessly. Was able to remove 'unknown' node and then re-add without issue. This wasn't exactly what I was looking for, but directed me to what solved my problem. Thank you.
 
Domenic said:
I just went through some self inflicted pain when I added a node to my cluster when one of my other nodes was offline. I didn't know that would cause so many problems. A warning in the UI would have been nice. The offline node when brought back online just didn't want to behave correctly. It hung in the shell about 30 seconds after boot and never attached to the cluster again. I could only SSH to it. I couldn't find a solution to fix its membership in the cluster. At this point it appeared best to remove it from the cluster. This was all on 8.2.x

From a good cluster node:

Code: 
pvecm nodes
pvecm delnode<nodename>

The node will still show as a ghost in the Datacenter view until the node is deleted from at least one nodes /etc/pve/nodes dir.
As long as there are files still in there, the node will show up in the UI.

Code: 
rm -r /etc/pve/nodes/<nodename>

On removed node to make it standalone and files editable:

Code: 
systemctl stop pve-cluster
/usr/bin/pmxcfs -l

rm -f /etc/pve/corosync.conf
rm -f /etc/corosync/corosync.conf
rm -f /etc/corosync/authkey
cd /etc/pve/nodes
rm -r <othernodename>  #for all other nodes

reboot now

Node is now standalone. The node can now be re-added to a cluster if desired.
Click to expand...
Thank You. That worked for me.
 
The correct way that I found to remove a node from a cluster is the following:

NOTE: for this case, we will remove node1 from a cluster. Also, Bash: blocks are the commands used, and Code: blocks are command outputs.


1- Use the following commands from one of the nodes 2 or 3 only

Bash: 
pvecm nodes

Code: 
root@node2:~# pvecm nodes

Membership information
----------------------
    Nodeid      Votes Name
         1          2 node1
         2          2 node2 (local)
         3          2 node3
root@node2:~#

Bash: 
pvecm delnode node1

Code: 
root@node2:~# pvecm delnode node1
Killing node 1
root@node2:~# pvecm nodes

Membership information
----------------------
    Nodeid      Votes Name
         2          2 node2 (local)
         3          2 node3
root@node2:~#


2- Remove all node1 entries from the config files

Bash: 
vi /etc/pve/ceph.conf
vi /etc/pve/corosync.conf


3- Remove old node1 directory

Bash: 
ls -l /etc/pve/nodes/node1/

Code: 
root@node2:~# ls -l /etc/pve/nodes/node1/
total 2
-rw-r----- 1 root www-data   83 Dec 31 08:26 lrm_status
drwxr-xr-x 2 root www-data    0 Mar 30  2024 lxc
drwxr-xr-x 2 root www-data    0 Mar 30  2024 openvz
drwx------ 2 root www-data    0 Mar 30  2024 priv
-rw-r----- 1 root www-data 1704 Mar 30  2024 pve-ssl.key
-rw-r----- 1 root www-data 1805 Mar 30  2024 pve-ssl.pem
drwxr-xr-x 2 root www-data    0 Mar 30  2024 qemu-server
root@node2:~#

Bash: 
rm -rfv /etc/pve/nodes/node1

Code: 
root@node2:~# rm -rfv /etc/pve/nodes/node1
removed directory '/etc/pve/nodes/node1/lxc'
removed '/etc/pve/nodes/node1/pve-ssl.key'
removed '/etc/pve/nodes/node1/lrm_status'
removed '/etc/pve/nodes/node1/pve-ssl.pem'
removed directory '/etc/pve/nodes/node1/priv'
removed directory '/etc/pve/nodes/node1/openvz'
removed directory '/etc/pve/nodes/node1/qemu-server'
removed directory '/etc/pve/nodes/node1'
root@node2:~#


I hope my findings above help others! :)
 
  • Like
Reactions: newuser2025
mira said:
Check if there are still leftover files in /etc/pve/nodes/<deleted-node>.
As long as there are files still in there, the node will show up in the UI.
Please make a backup of the files before you delete them, just to be safe!
Click to expand...
fahadshery said:
I have the same issue. I deleted a node but it's still visible in the GUI. Did you solve it or re-install is the only solution which is bonkers!!
Click to expand...
What ended up working for me is under /etc/pve/corosync.conf there was a second entry for node 2 once i removed that i didn't even need to restart proxmox.
 
proxwolfe said:
I, too, am trying to remove a node from my cluster (to set it up anew and rejoin it).

I managed to delete it using pvecm but it still showed up in the GUI.

So I removed the respective folder under /etc/pve/nodes and now it is gone.

But doing this, I discovered that I have another folder /etc/pve/nodes/pve which does not correspond to a node shown in the GUI or still in existence. My cluster has evolved over time and I removed all original nodes and replaced them with newer ones. It *may* be that one of the originals nodes was named "pve". Can I simply remove also this folder or might I kill my cluster?

Thanks
Click to expand...
Bro that dangling dead cluster was really bothering me in the gui - no clean install for me, removing the directory under `/etc/pve/nodes/` worked immediately.

I also have another directory under `/etc/pve/nodes` called `pve-proxmox`. This is a guess, but I believe that name may be left over from the initial installation based on the dates. It's probably safe to remove, but since I don't see it in the gui i'm leaving it.
 
Last edited:
You must log in or register to reply here.
Top Bottom