Proxmox Ve 2.2 2 node cluster: NODES restarting now and then!?

[danielmueller]

Dear all,
I have set up a proxmox 2 node cluster with IPMI fencing. My config:

<?xml version="1.0"?>
<cluster config_version="23" name="tropenklinik">
<cman expected_votes="1" keyfile="/var/lib/pve-cluster/corosync.authkey" two_node="1"/>
<fencedevices>
<fencedevice agent="fence_ipmilan" ipaddr="192.168.1.xxx" lanplus="1" login="XXXXX" name="ipmi1" passwd="XXXX" power_wait="5"/>
<fencedevice agent="fence_ipmilan" ipaddr="192.168.1.xxx" lanplus="1" login="XXXXX" name="ipmi2" passwd="XXXXX" power_wait="5"/>
</fencedevices>
<fence_daemon post_join_delay="60"/>
<clusternodes>
<clusternode name="proxmoxcl2" nodeid="1" votes="1">
<fence>
<method name="1">
<device name="ipmi2"/>
</method>
</fence>
</clusternode>
<clusternode name="proxmoxcl1" nodeid="2" votes="1">
<fence>
<method name="1">
<device name="ipmi1"/>
</method>
</fence>
</clusternode>
</clusternodes>
<rm>
<pvevm autostart="1" vmid="189"/>
</rm>
</cluster>Cluster Status for tropenklinik @ Thu Nov 29 09:22:08 2012
Member Status: Quorate
Member Name ID Status
------ ---- ---- ------
proxmoxcl2 1 Online, Local, rgmanager
proxmoxcl1 2 Online, rgmanager
Service Name Owner (Last) State
------- ---- ----- ------ -----
pvevm:189 proxmoxcl1 started



Every is working well, but now and then one of the cluster nodes get restarted without any reason!? As we plan to use the HA W2008 virtual machine on the proxmuxcluster as Database Server this is a no go. The clients working would
be cut off until the HA Va is started on the other node.
Is this behaviour normal proxmox feature or is there a way to stop it?
Cut of my /var/log/pve/tasks:
UPIDroxmoxcl2:0000231B:00086249:50B5F945:qmshutdown:189:root@pam: 1 50B5FA2A unexpected status
UPIDroxmoxcl2:000010A9:0003A32D:50B5ED1E:qmstart:189:root@pam: 1 50B5ED28 OK
UPIDroxmoxcl2:00000B18:0000172A:50B5E40A:startall::root@pam: 1 50B5E40A OK
UPIDroxmoxcl2:000010F3:000135FD:50B5E322:qmshutdown:189:root@pam: 1 50B5E40A unexpected status

Can someone give me a hint!?

Greetings
Daniel

 

[tom]

you need to check the syslog.

and also read the system requirements for HA, looks like you missed some parts.

http://pve.proxmox.com/wiki/High_Availability_Cluster

 

[jleg]

Dear all,
I have set up a proxmox 2 node cluster with IPMI fencing. My config:

you only have 2 nodes? How do you ensure having a quorum? you probably either need a 3rd node, or at least a sort of "fake quorum-only node" for HA...

 

[danielmueller]

cut of my syslog when restarting of one node:
Nov 29 09:56:04 proxmoxcl2 corosync[2014]: [TOTEM ] A processor joined or left the membership and a new membership was formed.
Nov 29 09:56:04 proxmoxcl2 kernel: dlm: closing connection to node 2
Nov 29 09:56:04 proxmoxcl2 rgmanager[2223]: State change: proxmoxcl1 DOWN
Nov 29 09:56:04 proxmoxcl2 corosync[2014]: [CPG ] chosen downlist: sender r(0) ip(192.168.134.131) ; members(old:2 left:1)
Nov 29 09:56:04 proxmoxcl2 pmxcfs[1844]: [dcdb] notice: members: 1/1844
Nov 29 09:56:04 proxmoxcl2 pmxcfs[1844]: [dcdb] notice: members: 1/1844
Nov 29 09:56:04 proxmoxcl2 corosync[2014]: [MAIN ] Completed service synchronization, ready to provide service.
Nov 29 09:56:04 proxmoxcl2 fenced[2071]: fencing node proxmoxcl1
Nov 29 09:56:55 proxmoxcl2 fenced[2071]: fence proxmoxcl1 success
Nov 29 09:56:56 proxmoxcl2 rgmanager[6233]: [pvevm] VM 189 is running
Nov 29 09:57:01 proxmoxcl2 rgmanager[6263]: [pvevm] VM 189 is running
Nov 29 09:57:39 proxmoxcl2 GlusterFS[3256]: [2012-11-29 09:57:39.772065] C [client-handshake.c:121:rpc_client_ping_timer_expired] 0-proxmox-share-client-1: server 192.168.1.201:24009 has not responded in the last 42 seconds, disconnecting.
Nov 29 09:57:39 proxmoxcl2 pvestatd[3138]: status update time (87.660 seconds)
Nov 29 09:57:41 proxmoxcl2 rgmanager[6296]: [pvevm] VM 189 is running

THe HA function of starting the virt. machine on the other node is working! But with delay!?
both nodes have their machine on a raid1 glusterfs volume. So all virt machines are identical!?
My clustat:
clustat
Cluster Status for tropenklinik @ Thu Nov 29 10:47:44 2012
Member Status: Quorate
Member Name ID Status
------ ---- ---- ------
proxmoxcl2 1 Online, Local, rgmanager
proxmoxcl1 2 Online, rgmanager
Service Name Owner (Last) State
------- ---- ----- ------ -----
pvevm:189 proxmoxcl2 started