Broken Cluster

[faisa7847]

Hi i removed a node form a cluster and the cluster broke

now i am try to add the node back in but having issues I am getting this error on pveproxy


pveproxy.service - PVE API Proxy Server
Loaded: loaded (/lib/systemd/system/pveproxy.service; enabled; preset: enabled)
Active: active (running) since Tue 2024-09-24 22:15:43 +03; 21s ago
Process: 1745 ExecStartPre=/usr/bin/pvecm updatecerts --silent (code=exited, status=0/SUCCESS)
Process: 1880 ExecStart=/usr/bin/pveproxy start (code=exited, status=0/SUCCESS)
Main PID: 1886 (pveproxy)
Tasks: 4 (limit: 309288)
Memory: 150.9M
CPU: 2.025s
CGroup: /system.slice/pveproxy.service
├─1886 pveproxy
├─1998 "pveproxy worker"
├─1999 "pveproxy worker"
└─2000 "pveproxy worker"

Sep 24 22:16:03 prx1 pveproxy[1886]: starting 2 worker(s)
Sep 24 22:16:03 prx1 pveproxy[1886]: worker 1998 started
Sep 24 22:16:03 prx1 pveproxy[1886]: worker 1999 started
Sep 24 22:16:03 prx1 pveproxy[1963]: worker exit
Sep 24 22:16:03 prx1 pveproxy[1998]: /etc/pve/local/pve-ssl.key: failed to load local private key (key_file or key) at /usr/share/perl5/PVE/APIServer/AnyEvent.pm l

 

[dherzig]

Hi faisa7847!

Reading up the docs you're not expected to plug out a server from a cluster and then plug it in again -- you'll have to put PVE again onto the given server and then do the join.

There's this in the bottom of the section about removing a node from the cluster [0]:

If, for whatever reason, you want this server to join the same cluster again, you have to:

• do a fresh install of Proxmox VE on it,
• then join it, as explained in the previous section.

Also, if you use the same IP for it after reinstallation, you'll have to run

pvecm updatecerts

on it to update the ssh-fingerprints for the remaining nodes of the cluster.

[0] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_remove_a_cluster_node

 

[faisa7847]

Well i had 4 nodes i removed the fouth node and the first node also came out don't know why

 

[dherzig]

Could you post the output of `pvecm status` from one of the working nodes?