Node Stuck on GUI Cant Remove

[Epicstockmedia]

hey all!

I was trying to add another box to my clusters but I messed up the addressing and caused it to fail but now i cant remove it.

Is this something that can be manually removed?

Thank you !

 

[justinclift]

Ahhh. That phantom node will probably be showing up due to still having a structure for it under /etc/pve/nodes/.

It's fairly standard for that to happen when deleting a node from a cluster.

If you've already removed the node from the cluster fully (ie pvecm delnode flash-pve-a), then you can delete the flash-pve-a directory from under /etc/pve/nodes, then reload your web interface.

It should be gone at that point. Hmm... actually, you might need to log out of the web interface and then back in. (It's been a while since I last needed to do it.)

---

Note that if you haven't deleted the node first using pvecm delnode, then something else is going on so deleting stuff from /etc/pve/nodes is probably not the right direction to take.

 

[Epicstockmedia]

Sorry for late reply.

So From the GUI if I click the node i get



When I'm data center level and have clicked cluster it see this

/etc/pve/nodes/flash-pve-a/pve-ssl.pem' does not exist! (500)




I did mess up for adding the right IP address. prolly messed up.

 

[justinclift]

In the GUI, if you open up a shell session (specifically in the GUI), then run ping flash-pve-a does it seem to correctly find an address and ping it, or does it give an error message?

Hmmm, might be better to screenshot the output and paste it here?

 

[Epicstockmedia]

This is from the flash A - looks like the file etc/pve/corosync.conf is empty.



This is one from node 2in the cluster.



When i try to connect to the node. i cant get to the shell in gui of the Cluster I tried adding the node too.



Also For clarity.. I would 100% want to remove flashA node this cluster. Would i need to remove them from the etc/pve/corosync.conf on each node?

 

[Epicstockmedia]

 

[justinclift]

k, so that pvecm status command that you ran on flash-pve-a is a decent idea, and it shows that it's not part of any cluster. The /etc/pve/corosync.conf file gets created as part of the process of creating or joining a Proxmox cluster.

Now, lets back to the previous thing to try. On the prox-clus-1 node, please start a gui shell and run ping -c 3 flash-pve-a then paste the output here.

What that should do is let us know if prox-clus-1 is able to resolve the "flash-pve-a" name. It sounds like it probably can't.

It's probably a decent idea to run ip a on both prox-clus-1 and flash-pve-a and paste the output from them here as well. It's ok to run that via ssh, it doesn't have to be done via the gui shell thing.

The ip a command will show us the ip address configuration you're using for both of those hosts, which could be useful just in case there's something unexpected/weird with the ip addressing.

 

[justinclift]

Oh. Just now it sunk in that you have the IP addresses already showing in corosync.conf from node 2.

So:

• flash-pve-a : 10.0.2.88
• prox-clus-1 : 10.0.99.91
• prox-clus-2 : 10.0.99.92
• prox-clus-3 : 10.0.99.93

From that, my first guess is there's probably something wrong with the netmask or CIDR for your network nodes. It's probably 255.255.255.0 (also known as /24), when it should be 255.255.0.0 (also known as /16) for the above addressing to work.

That's just a first guess though. The ip a output from both flash-pve-a and prox-clus-1 would tell us for sure.

 

[justinclift]

Would i need to remove them from the etc/pve/corosync.conf on each node?

As a data point, the entire /etc/pve directory on the cluster nodes is a special type of thing. Any changes made in that directory get automatically pushed to the other cluster nodes.

So if you change that file on one node, then it'll automatically be pushed to the other nodes within a second or so. ie. you only need to change it on one node for them to all have it

To remove the broken entry from the cluster configuration, you're probably best to run pvecm delnode flash-pve-a first and see if that works.

Manually mucking around with that file can be done later on, if the command doesn't work.

 

[Epicstockmedia]

please start a gui shell and run ping -c 3 flash-pve-a then paste the output here.


please start a gui shell and run ping -c 3 flash-pve-a then paste the output here.

 

[Epicstockmedia]

Ok so just saw your other posts! thansk for the details and thoughts!

I just ran "pvecm delnode flash-pve-a" worked after a browser refresh



Fantastic!

THANK YOU @justinclift!

 

[justinclift]

Awesome! Sounds like things are good again now.

For adding that flash-pve-a node again at some point, I'd probably try setting its ip address to be in the 10.0.99.x network range to minimise potential problems. Maybe 10.0.99.94, depending on whether that's already taken or not.