waiting for quorum, cluster setup problem

[Tom Behets]

Hi!

I am struggling with setting up my proxmox cluster.

First, i created the cluster configuration with the following command:

pvecm create proxhost

I don't have the result anymore, but i remember it failed after it was waiting for quorum.
When i retry the same i get

cluster config '/etc/pve/cluster.conf' already exists

OK, so i imagine this would eventually work anyhow.

To see the nodes, i used

pvecm nodes

This gives me

cman_tool: cman_get_node_count failed: Node is not yet a cluster member

I add the node with

pvecm addnode proxhost

this gives me

cluster not ready - no quorum?

So, what is this quorum? And i don't see a way to remove the cluster configuration and try again?
Is there anything i could do now to get this going?

Many thanks!
Tom

 

[dietmar]

So, what is this quorum

http://en.wikipedia.org/wiki/Quorum_(distributed_computing)

What is the output of

# /etc/init.d/pve-cluster restart

and

# cat /etc/pve/cluster.conf

 

[Tom Behets]

# /etc/init.d/pve-cluster restart

Restarting pve cluster filesystem: pve-cluster.

Cluster restarted fine i guess.

# cat /etc/pve/cluster.conf

<?xml version="1.0"?><cluster name="proxhost" config_version="1">


  <cman keyfile="/var/lib/pve-cluster/corosync.authkey">
  </cman>


  <clusternodes>
  <clusternode name="proxhost" votes="1" nodeid="1"/>
  </clusternodes>


</cluster>

thanks for your help

 

[dietmar]

Looks OK. What is the output of:

# /etc/init.d/cman start

 

[Tom Behets]

Starting cluster:    Checking if cluster has been disabled at boot... [  OK  ]
   Checking Network Manager... [  OK  ]
   Global setup... [  OK  ]
   Loading kernel modules... [  OK  ]
   Mounting configfs... [  OK  ]
   Starting cman... Corosync Cluster Engine is already running
[FAILED]

 

[dietmar]

OK, can you try:

# /etc/init.d/cman stop
# /etc/init.d/cman start

 

[Tom Behets]

root@proxhost ~ # /etc/init.d/cman stop
Stopping cluster:
Stopping dlm_controld... [ OK ]
Stopping fenced... [ OK ]
Stopping cman... [ OK ]
Unloading kernel modules... [ OK ]
Unmounting configfs... [ OK ]
root@proxhost ~ # /etc/init.d/cman start
Starting cluster:
Checking if cluster has been disabled at boot... [ OK ]
Checking Network Manager... [ OK ]
Global setup... [ OK ]
Loading kernel modules... [ OK ]
Mounting configfs... [ OK ]
Starting cman... Corosync Cluster Engine is already running
[FAILED]

 

[dietmar]

What happens when you reboot that node?

 

[Tom Behets]

Same problem, says node is not a cluster member yet.
And when i add the node, 'cluster not ready, no quorum'. :'(

 

[tom]

your switches supports IP multicast?

 

[Tom Behets]

I have no idea. My server is hosted at hetzner.de
But the problem of adding the node to itself, can this be a switch problem?
I will contact hetzner to ask if the switch supports multicast

 

[tom]

See also http://pve.proxmox.com/wiki/Proxmox_VE_2.0_Cluster#Requirements

 

[Tom Behets]

Got answer from hetzner, they dont allow multicast on their servers.

Maybe it's a good idea to start a new wiki page with this information?
A list of hosts with their shortcomings, if any. Or confirmed hosts with no trouble.
Just an idea...

Thanks Tom & Dietmar for your help, much appreciated.

 

[net-4-it]

<cluster name="[B]proxhost[/B]" config_version="1">
<clusternode name="[B]proxhost[/B]" votes="1" nodeid="1"/>

thanks for your help[/QUOTE]


It's possibly a problem, if you use the same name for the cluster as the name of a node in cluster. Try to take an other name for the cluster.
At this moment the system, where you you have initialized the cluster, is already a member of that cluster, so you don't have to add this node again.

 

[ksosez]

I just found out singlehop doesnt support multicast. Now I am not sure what I am going to do in terms of upgrading since I cant use clustering like I did before.


Anyone got any thoughts?

 

[e100]

Anyone got any thoughts?

Two thoughts:

The issue here is that the hosts prevent using multicast on their switches.
Maybe they can give you a dedicated switch on a 2nd port of the servers you have and do multicast over that switch.
Likely a stretch but some hosts might be willing to do that.

My next thought is to create a tunnel between your servers and send the multicast through the tunnel.
This seems to describe such a setup: http://linux-ip.net/gl/ip-tunnels/node9.html

Edit:
I found this example for Ubuntu, should work in debian too:
http://onvox.net/linux/how-to-create-ipv4-gre-tunnels-in-ubuntu

You would need to get the tunnel working.
Then edit /etc/hosts and set the names for your nodes to <tunnel IP> since corosync uses multicast on the IP based on /etc/hosts file.

 

[ksosez]

Just to reply to my own question:

I ran across this which talks about using corosync in places where they dont support multicast. (Amazon EC2 and most dedicated hosting companies dont seem to)

http://serverfault.com/questions/276170/alternatives-to-heartbeat-pacemaker-and-corosync

I am going to have to setup some test systems and see if it works after editing that file or whether proxmox wont like that.

 

[e100]

ksosez asked some questions in another thread and I believe he has helped us find a way to get the cluster working without multicast!

http://forum.proxmox.com/threads/8204-Understanding-cluster-conf

 

[Nelson]

ksosez asked some questions in another thread and I believe he has helped us find a way to get the cluster working without multicast!

http://forum.proxmox.com/threads/8204-Understanding-cluster-conf

I have this same problem with dedicated server in OVH, but I dont tray again the solution with udpu/unicast in cluster.conf, maybe it's working !?


<cman transport="udpu" keyfile="/var/lib/pve-cluster/corosync.authkey">If this last solution dont work, the unique solution is private switch or private rack with the same ip ranges!