Error fencing proxmox cluster node

[jaraigon]

Hi all,

I have a three nodes proxmox cluster with HA (iSCSI shared storage). I've followed proxmox wiki articles for doing so and main things, such as live migration of virtual machines or automatic restarting of a HA enabled virtual machine in other node when the I provoke a crash in the node in which it was running, work great.

However, I'm not sure if my fencing configuration wil work properly because of the results of manually fencing nodes. I've tried to fence every node from each other with the next results:

Fencer/Fencee	proxmox-n1	proxmox-n2	proxmox-n3
proxmox-n1	Not Needed	OK	OK
proxmox-n2	OK	Not Needed	OK
proxmox-n3	FAIL	FAIL	Not Needed

I'm using IPMI in nodes 1 and 2 and iDRAC v6 in node 3. And this is my cluster.conf fencing configuration:

<fencedevices>
<fencedevice agent="fence_ipmilan" name="ipmilan_proxmox-n1" ipaddr="192.168.0.1" login="root" passwd="Password" power_wait="5"/>
<fencedevice agent="fence_ipmilan" name="ipmilan_proxmox-n2" ipaddr="192.168.0.2" login="root" passwd="Password" power_wait="5"/>
<fencedevice agent="fence_idrac" name="idrac_proxmox-n3" ipaddr="192.168.0.3" login="root" passwd="Password" power_wait="5"/>
</fencedevices>

<clusternodes>
<clusternode name="proxmox-n1" votes="1" nodeid="1">
<fence>
<method name="1">
<device name="ipmilan_proxmox-n1"/>
</method>
</fence>
</clusternode>
<clusternode name="proxmox-n2" votes="1" nodeid="2">
<fence>
<method name="1">
<device name="ipmilan_proxmox-n2"/>
</method>
</fence>
</clusternode>
<clusternode name="proxmox-n3" votes="1" nodeid="3">
<fence>
<method name="1">
<device name="idrac_proxmox-n3"/>
</method>
</fence>
</clusternode>
</clusternodes>

When I run fence_node command in node 3 I get this output:

root@proxmox-n3:~# fence_node proxmox-n1 -vv
fence poulsen dev 0.0 agent fence_ipmilan result: error from agent
agent args: nodename=poulsen agent=fence_ipmilan ipaddr=192.168.0.1 login=root passwd=Password power_wait=5
fence proxmox-n1 failed

But in Syslog tab on web interface the next messages are shown and node-1 is actually fenced:

Jun 14 12:18:47 proxmox-n3 fenced[3371]: fencing node proxmox-n1
Jun 14 12:19:37 proxmox-n3 fenced[3371]: fence proxmox-n1 success

So my question is, can I safety ignore the fence_node error output?


Thanks in advance

 

[tom]

you posted 4 times your issue - this is NOT needed.

(post from new users are moderated, you will see a short message after you submit your thread).

 

[jaraigon]

you posted 4 times your issue - this is NOT needed.

(post from new users are moderated, you will see a short message after you submit your thread).

Yes, I'm sorry so much. I've realised it just with the last try to post. I didn't know than posts were moderated. The message only lasts a copule of seconds and I hadn't time to read it.

 

[dietmar]

So my question is, can I safety ignore the fence_node error output?

Looks you run into a timeout. I would simply try to increase the timeout.

 

[jaraigon]

Looks you run into a timeout. I would simply try to increase the timeout.

It seems a good idea but, how can I do that? I've read fence_node and cluster.conf man pages but I haven't found anything about handling timeout.

 

[dietmar]

See:

# man fence_ipmilan

You can simply add a 'timeout=X' parameter to the fencedevice section in cluster.conf.

 

[jaraigon]

See:

# man fence_ipmilan

You can simply add a 'timeout=X' parameter to the fencedevice section in cluster.conf.

Thanks again for your help.

By making a lot of tests (including adding different values for timeout parameter) I've realised that fence_node command has not a regular behaviour in my cluster. Independently of what node runs the command, sometimes it fails and sometimes it works.

I've been reading a lot documentation and forums and I think that the problem may be in using the same integrated network interface card for IPMI as part of an active/pasive bonding configuration. So I'll try to install an additional NIC for bonding purposes and use the IPMI integrated NIC for IPMI operations only.

I'll make you know if this is finally the solution.