C
Chris Rivera
Guest
Lately my weekly issue has become something different.... the cluster is up and running but split.
Half of the nodes are in sync with some nodes while the other nodes are in sync with the other nodes.
How do i send a flush command to drop the current cluster config and pull the latest config file from a specific node?
This is the 3rd time in 3 weeks this has happened. This happens after we have an abusive VPS user on the cloud which does something to make cman fail on all nodes. Once we find this vm and bring it offline the cluster and cman start to work again but in split cluster. Rebooting a node will let me know which cluster is the latest be cause it will join that cluster... then rebooting all the nodes on the cluster that the first rebooted node didn't join will solve the problem.
How do i solve this without restarting 6 nodes?
The 6 nodes that we have to reboot will not stop cman
service cman restart fails on stop.... on the other cluster cman is up and running and is able to be restarted, but since it has 5 nodes instead of 6 there is no quorum on the live running cluster.
Half of the nodes are in sync with some nodes while the other nodes are in sync with the other nodes.
How do i send a flush command to drop the current cluster config and pull the latest config file from a specific node?
This is the 3rd time in 3 weeks this has happened. This happens after we have an abusive VPS user on the cloud which does something to make cman fail on all nodes. Once we find this vm and bring it offline the cluster and cman start to work again but in split cluster. Rebooting a node will let me know which cluster is the latest be cause it will join that cluster... then rebooting all the nodes on the cluster that the first rebooted node didn't join will solve the problem.
How do i solve this without restarting 6 nodes?
The 6 nodes that we have to reboot will not stop cman
service cman restart fails on stop.... on the other cluster cman is up and running and is able to be restarted, but since it has 5 nodes instead of 6 there is no quorum on the live running cluster.