[SOLVED] Remove or reset cluster configuration.

T

taurolyon

Member
Sep 7, 2021
9
1
8
I have 2 PVE nodes that I would like to setup as a cluster, however both have configured clusters and I can't figure out how to remove the cluster from one to join the other.
 
  • Like
Reactions: anturk
Thank you! This still appears to be the way to clear it! EDIT: Not quite there (see next post).

When I attempted this earlier, I must have missed the pmxcfs -l line to force local mode.

Code: 
         .://:`              `://:.            root@pve
       `hMMMMMMd/          /dMMMMMMh`          --------
        `sMMMMMMMd:      :mMMMMMMMs`           OS: Proxmox VE 7.2-7 x86_64
`-/+oo+/:`.yMMMMMMMh-  -hMMMMMMMy.`:/+oo+/-`   Host: PowerEdge R710
`:oooooooo/`-hMMMMMMMyyMMMMMMMh-`/oooooooo:`   Kernel: 5.15.39-4-pve
  `/oooooooo:`:mMMMMMMMMMMMMm:`:oooooooo/`     Uptime: 4 mins
    ./ooooooo+- +NMMMMMMMMN+ -+ooooooo/.       Packages: 746 (dpkg)
      .+ooooooo+-`oNMMMMNo`-+ooooooo+.         Shell: bash 5.1.4
        -+ooooooo/.`sMMs`./ooooooo+-           Terminal: /dev/pts/0
          :oooooooo/`..`/oooooooo:             CPU: Intel Xeon X5670 (24) @ 2.927GHz [36.0°on]
          :oooooooo/`..`/oooooooo:             GPU: NVIDIA Quadro P620
        -+ooooooo/.`sMMs`./ooooooo+-           Memory: 19.96GiB / 141.32GiB (14%)
      .+ooooooo+-`oNMMMMNo`-+ooooooo+.
    ./ooooooo+- +NMMMMMMMMN+ -+ooooooo/.
  `/oooooooo:`:mMMMMMMMMMMMMm:`:oooooooo/`
`:oooooooo/`-hMMMMMMMyyMMMMMMMh-`/oooooooo:`
`-/+oo+/:`.yMMMMMMMh-  -hMMMMMMMy.`:/+oo+/-`
        `sMMMMMMMm:      :dMMMMMMMs`
       `hMMMMMMd/          /dMMMMMMh`
         `://:`              `://:`

root@pve:~# systemctl stop pve-cluster corosync
root@pve:~# pmxcfs -l
[main] notice: forcing local mode (although corosync.conf exists)
root@pve:~# tar -zcvf etc_corosync.tar.gz /etc/corosync/*
tar: Removing leading `/' from member names
/etc/corosync/authkey
tar: Removing leading `/' from hard link targets
/etc/corosync/corosync.conf
root@pve:~# rm /etc/corosync/*
root@pve:~# tar -zcvf corosync_conf.tar.gz /etc/pve/corosync.conf
tar: Removing leading `/' from member names
/etc/pve/corosync.conf
root@pve:~# rm /etc/pve/corosync.conf
root@pve:~# killall pmxcfs
root@pve:~# systemctl start pve-cluster
root@pve:~# pvecm status
Error: Corosync config '/etc/pve/corosync.conf' does not exist - is this node part of a cluster?
root@pve:~#
 
Last edited:
Spoke too soon. While the cluster information is removed; when attempting to join the cluster, I get the following:
Code: 
detected the following error(s):
* this host already contains virtual guests
TASK ERROR: Check if node may join a cluster failed!

I attempted the following:
Code: 
root@pve:~# pvecm nodes
Error: Corosync config '/etc/pve/corosync.conf' does not exist - is this node part of a cluster?
root@pve:~# pvecm delnode pve
Cannot delete myself from cluster!

I'm stumped. o_O
 
  • Like
Reactions: taurolyon
Created acct just to reply here. I'm not OP but I had this work for me on 2 nodes that wouldn't communicate. Only thing is for the second node I needed to remove the folder /etc/pve/nodes/<NODE NAME> for it to remove from the Web UI, otherwise it looks good. I believe this has also worked for me in the past with these same 2 nodes.
 
  • Like
Reactions: taurolyon and tbrooks
**cough** what If someone deleted the /etc/pve/nodes/YYYY folder?

I did this and it took the proxmox server, effectively offline. The GUI will not load, cannot reach it via SSH (it is on the network). Looking at another unrelated server, the only files of importance appears to be the pve-ssk.key & PEM file.

I rebooted the box, no change. Will this constitute a reload of the OS? and a firm kick in the rear for myself?
 
You probably need to restart the services you stopped during your attempt to delete the node info.

systemctl start pve-cluster.service
systemctl start pvestatd.service
systemctl start pvedaemon.service

I got that from here: https://forum.proxmox.com/threads/removing-deleting-a-created-cluster.18887/post-154254

Tested a matter of minutes ago. I caused the problem by killing those services through the web GUI rather than a local shell or SSH, perhaps. Restarting the services fixed the problem, no reboot needed.
 
  • Like
Reactions: taurolyon and stevensedory
Hey ! I would like to give another answer, this worked for me in V 8.x.x,
thanks for the base script !:)


Bash: 
systemctl stop pve-cluster corosync
pmxcfs -l
rm -R /etc/corosync/*
rm -R /etc/pve/nodes
killall pmxcfs
systemctl start pve-cluster
 
  • Like
Reactions: taurolyon
The above worked for me as well to remove my node from a cluster. BUT It looks like "rm -R /etc/pve/nodes" must be overkill as when I now try to add my node to a different cluster I get the error "'/etc/pve/nodes/pve01/pve-ssl.pem' does not exist! (500)" :-(

Not sure if I can copy from another copy of pve; guessing not as it's probably generated per machine. I might have to reinstall...

EDIT: Did some more Googlefu and after a few things that didn't work, this basically solved my problem:
J

[SOLVED] Thread 'No quorum error'

Hi all.

I was having a cluster composed by two VE servers.

One of the servers suffer a HD problem and died.

Now I'm blocked in a lot of operation wit error "cluster not ready - no quorum?"

I have replaced the HD in the failing cluster and reinstalled proxmox, but when trying to join the cluster I get the same error.

Is there a way to avoid the no quorum error or to manually remove the old failed server from the cluster?

Thanks in advance.
 
Last edited:
hi guys,
newbie here what is the difference between the commands listed in this thread vs pvecm delnode command? I just created a 3 node cluster and now am trying to learn how to remove a node from the cluster and saw this but on a youtube video saw someone use the pvecm delnode command to remove the node.

thank you in advaced.
 
newuser2025 said:
hi guys,
newbie here what is the difference between the commands listed in this thread vs pvecm delnode command? I just created a 3 node cluster and now am trying to learn how to remove a node from the cluster and saw this but on a youtube video saw someone use the pvecm delnode command to remove the node.

thank you in advaced.
Click to expand...
delnode is to remove a node

this thread is about to delete / remove / close the full cluster , not just remove one node
 
FuxT said:
Hey ! I would like to give another answer, this worked for me in V 8.x.x,
thanks for the base script !:)


Bash: 
systemctl stop pve-cluster corosync
pmxcfs -l
rm -R /etc/corosync/*
rm -R /etc/pve/nodes
killall pmxcfs
systemctl start pve-cluster
Click to expand...
Just so anyone who else stumbles on this from the internet like I did after botching my first attempt to create a cluster - DO NOT USE THIS SCRIPT. The line "rm -R /etc/pve/nodes" will delete all of your VMs and LXCs.
 
Impact said:
/etc/pve should be empty anyways if pve-cluster/pmxcfs is stopped.
Click to expand...
no it is not, because the line before it is "pmxcfs -l" which will start a local instance of pmxcfs.. yes, that command is dangerous and likely not needed (unless you want to clear out all node-specific state, including of the node you are running it on!), but so is the whole procedure.
 
You must log in or register to reply here.
Top Bottom