Problem with quorum disk

[TUM]

Hi all,
I've taken over an Proxmox cluster with 3 nodes and I've got an problem with the configuration of the quorum disk.
Proxmox version:
Node 1: 3.0
Node 2: 3.0 RC2
Node 3: 3.0

I've followed the instruction -> http://pve.proxmox.com/wiki/Two-Node_High_Availability_Cluster and added the qdisk to the cluster.
When i started the cluster "/etc/init.d/cman reload" on Node 1, I've got following result:

root@node1:~# /etc/init.d/cman reload
Stopping cluster:
Leaving fence domain... [ OK ]
Stopping dlm_controld... [ OK ]
Stopping fenced... [ OK ]
Stopping qdiskd... [ OK ]
Stopping cman... [ OK ]
Waiting for corosync to shutdown:[ OK ]
Unloading kernel modules... [ OK ]
Unmounting configfs... [ OK ]
Starting cluster:
Checking if cluster has been disabled at boot... [ OK ]
Checking Network Manager... [ OK ]
Global setup... [ OK ]
Loading kernel modules... [ OK ]
Mounting configfs... [ OK ]
Starting cman... tempfile:29: element fence: Relax-NG validity error : Expecting element clusternode, got fence
tempfile:14: element clusternode: Relax-NG validity error : Element clusternodes has extra content: clusternode
Configuration fails to validate
[ OK ]
Starting qdiskd... [ OK ]
Waiting for quorum... [ OK ]
Starting fenced... [ OK ]
Starting dlm_controld... [ OK ]
Tuning DLM kernel config... [ OK ]
Unfencing self... [ OK ]
Joining fence domain... [ OK ]

But I've didn't get the gdisk running on Node 2 or Node 3

root@node3:~# /etc/init.d/cman reload
Stopping cluster:
Stopping dlm_controld... [ OK ]
Stopping fenced... [ OK ]
Stopping qdiskd... [ OK ]
Stopping cman... [ OK ]
Waiting for corosync to shutdown:[ OK ]
Unloading kernel modules... [ OK ]
Unmounting configfs... [ OK ]
Starting cluster:
Checking if cluster has been disabled at boot... [ OK ]
Checking Network Manager... [ OK ]
Global setup... [ OK ]
Loading kernel modules... [ OK ]
Mounting configfs... [ OK ]
Starting cman... tempfile:29: element fence: Relax-NG validity error : Expecting element clusternode, got fence
tempfile:14: element clusternode: Relax-NG validity error : Element clusternodes has extra content: clusternode
Configuration fails to validate
[ OK ]
Starting qdiskd...
[FAILED]

The cluster.conf version is the same, so I don't know where the problem is.


Any help is much appreciated
Regards
Markus

 

[dietmar]

Please post your cluster.conf file (seems to contain syntax errors).

 

[TUM]

Hello Dietmar,
here is our cluster.conf file.

<?xml version="1.0"?>
<cluster config_version="33" name="tusp">
<cman expected_votes="3" keyfile="/var/lib/pve-cluster/corosync.authkey"/>
<fencedevices>
<fencedevice agent="fence_ipmilan" ipaddr="10.162.192.5" lanplus="1" login="admin" name="ipmi1" passwd="XXX" power_wait="5"/>
<fencedevice agent="fence_ipmilan" ipaddr="10.162.192.6" lanplus="1" login="admin" name="ipmi2" passwd="XXX" power_wait="5"/>
</fencedevices>
<quorumd allow_kill="0" interval="3" label="proxmox1_qdisk" tko="10">
<heuristic interval="3" program="ping $GATEWAY -c1 -w1" score="1" tko="4"/>
<heuristic interval="3" program="ip addr | grep eth1 | grep -q UP" score="2" tko="3"/>
</quorumd>
<totem token="54000"/>
<clusternodes>
<clusternode name="node1" nodeid="1" votes="1">
<fence>
<method name="1">
<device name="ipmi1"/>
</method>
</fence>
</clusternode>
<clusternode name="node2" nodeid="2" votes="1">
<fence>
<method name="1">
<device name="ipmi2"/>
</method>
</fence>
</clusternode>
<clusternode name="node3" nodeid="3" votes="1"/>
</clusternodes>
<rm/>
</cluster>

Regards
Markus

 

[dietmar]

Oh, this is a 3-node cluster, but only 2 fence devices? And you did not configure fencing for node3? Also, qdisk is normally not used on 3-node clusters (not needed)

 

[TUM]

Hi Dietmar,
node3 hasn't got an remote console, so I didn't know (at the moment) how to implement fencing on this node. Did you got an advice for me?
Something like this?

<clusternode name="node3" votes="1" nodeid="3">
<fence>
<method name="1">
<device name="node3" action="reboot"/>
</method>
</fence>
</clusternode>

Ok than i'll forget the configuration of gdisk and I'll forward with DRBD.

Regards
Markus