[SOLVED] Node not connecting to cluster

losarbitan

New Member
Nov 23, 2023
19
2
3
Hello all,

Recently I downsized my 4 node cluster to just 2 nodes. I had some free IP's now and wanted to switch my secondary node (SystemB) from 10.11.12.108 to 10.11.12.104. Whilst my main node (SystemA) lives on 10.11.12.105. I did not read up on correct procedure for changing the IP of nodes and messed up a bit. I edited the /etc/hosts and /etc/network/interfaces file on SystemB and ran an ifreload -a to update the changes. I also changed the hostname from SystemB to System2. I then noticed the cluster issues. Subsequently I decided to revert these changes so that I could use proper procedure to edit the IP of a node. However, after reverting the changes back to the original state the nodes no longer seem to be able to connect to each other. Actually I tried this twice and the cluster issues only started on the second time of trying this, I do not remember exactly what changed between the first and second time I tried it. I think I only changed the /etc/hosts file the first time.

I have checked and the /etc/corosync/corosync.conf and /etc/pve/corosync.conf files are all identical to each other on both nodes. I have tried many of the proposed solutions that I could find on the forums here. I think I did not do anything lasting to my configs anyway. All permanent changes that I did are (I think):
  1. Increased the quorum vote of SystemA from 1 to 2, I think I did not increment the config version, but this was before I had cluster issues (I think).
  2. Upped the corosync conf files to a version number higher than previously tested (version: 7)
  3. started up local mode and this created 2 pmxcfs processes (I think) that I now can not seem to get rid off when I run a killall $(pidof pmxcfs), this just returns the pid's followed by no process found.
Attached to the post are the config files I was talking about. Since I can only add 10 files the output of status commands are in the post.

Thanks in advance for any help!
 
Last edited:
Last edited:
  • Like
Reactions: leesteken