Lost / bad node

[sean]

I have a node that is not behaving and I seem to have broken the connection or something has gone wrong with the node and any advice or suggestions would be appreciated.

From the clip below you can see that the third node shows as red and the machine names are only presented as id's. I have tried to run pvecm from a good node to delete the bad node but the node remains.

 

[sean]

Apr 19 14:00:56 dataq1 pmxcfs[39357]: [status] crit: cpg_send_message failed: 9

Any idea what this means?

 

[dietmar]

Apr 19 14:00:56 dataq1 pmxcfs[39357]: [status] crit: cpg_send_message failed: 9

That means cman is not running. What is the output of

# /etc/init.d/cman start

on that node?

 

[marotori]

I have been helping Sean fix this.

Minor nightmare

The corosync cluster pretty much corrupted itself. I have had to hack around - but have successfully deleted the node from the cluster.. removed the cluster config; then added the 'clean' node back to the cluster.

How.. I have no idea! Pretty much 100 million error messages and a bit of luck. But all working now.

Rob