DRDB and fencing on IBM servers

decibel83

Active Member
Oct 15, 2008
178
1
38
Thank you very much for your answer.
Could you explain me how that device could help me for fencing features, please?
 

snpz

Member
Mar 18, 2013
20
0
21
Use the Management Interface as a fence device. This is what I use in my clusters with no issues.
Hi, there!
Could you give a hint how to do that?!
I have 3 IBM X3550 servers with Proxmox 2.3 configured in cluster, but i would like to enable HA as well using SM interface.
Thanks in advance.
 

snpz

Member
Mar 18, 2013
20
0
21
So i have to install ipmitool on all nodes?! But how to configure this SM interface to have an IP, username and passwd?!
Sorry about those stupid questions, but i just want to get this thing working!
 

adamb

Renowned Member
Mar 1, 2012
1,163
44
68
So i have to install ipmitool on all nodes?! But how to configure this SM interface to have an IP, username and passwd?!
Sorry about those stupid questions, but i just want to get this thing working!

Its the IMM, the IP is configured in the bios of your machine. The IMM must be connected to your network.
 

snpz

Member
Mar 18, 2013
20
0
21
Its the IMM, the IP is configured in the bios of your machine. The IMM must be connected to your network.
It's the first time i'm trying to use SM interface. In BIOS configured BMC interfaces for all nodes. Each has their IP address, subnet mask, gateway, user and password. All nodes has a patch cord to switch, but there is no indication of any kind of activity on that interface.
Ok, installed ipmitool on all nodes. Created cluster.conf.new, but it doesnt validate :(
Code:
root@vm1:~# ccs_config_validate -v -f /etc/pve/cluster.conf.new
Creating temporary file: /tmp/tmp.fWA7S65o2S
Config interface set to:
Configuration stored in temporary file
Updating relaxng schema
Validating..
tempfile:13: element fence: Relax-NG validity error : Expecting element clusternode, got fence
tempfile:12: element clusternode: Relax-NG validity error : Element clusternodes has extra content: clusternode
Configuration fails to validate
Validation completed
 

snpz

Member
Mar 18, 2013
20
0
21
Ok, tested my settup with:
Code:
fence_ipmilan -l User -p Passwd -P -a 10.10.10.6 -T 4 -o off -v
Server goes down. With:
Code:
fence_ipmilan -l User -p Passwd -P -a 10.10.10.6 -T 4 -o on -v
Server comes alive :)
My cluster.conf.new looks like this, but it doesnt go through validation :(
Code:
<?xml version="1.0"?><cluster name="cluster" config_version="4">


  <cman keyfile="/var/lib/pve-cluster/corosync.authkey">
  </cman>
  <fencedevices>
        <fencedevice agent="fence_ipmilan" name="ipmi1" lanplus="1" ipaddr="10.10.10.4" login="User" passwd="Passwd" power_wait="5"/>
        <fencedevice agent="fence_ipmilan" name="ipmi2" lanplus="1" ipaddr="10.10.10.5" login="User" passwd="Passwd" power_wait="5"/>
        <fencedevice agent="fence_ipmilan" name="ipmi3" lanplus="1" ipaddr="10.10.10.6" login="User" passwd="Passwd" power_wait="5"/>
  </fencedevices>
  <clusternodes>
  <clusternode name="vm1" votes="1" nodeid="1"/>
        <fence>
            <method name="1">
                 <device name="ipmi1"/>
            </method>
        </fence>
  <clusternode name="vm2" votes="1" nodeid="2"/>
        <fence>
            <method name="1">
                 <device name="ipmi2"/>
            </method>
        </fence>
  <clusternode name="vm3" votes="1" nodeid="3"/>
         <fence>
            <method name="1">
                 <device name="ipmi3"/>
            </method>
        </fence>
  </clusternodes>


</cluster>

Couldn't find out what does these lines are for in IPMI example:
Code:
[COLOR=#000000][B]<rm[/B][/COLOR][COLOR=#000000][B]>[/B][/COLOR]
    [COLOR=#009900][COLOR=#000000][B]<service[/B][/COLOR] [COLOR=#000066]autostart[/COLOR]=[COLOR=#FF0000]"1"[/COLOR] [COLOR=#000066]exclusive[/COLOR]=[COLOR=#FF0000]"0"[/COLOR] [COLOR=#000066]name[/COLOR]=[COLOR=#FF0000]"ha_test_ip"[/COLOR] [COLOR=#000066]recovery[/COLOR]=[COLOR=#FF0000]"relocate"[/COLOR][COLOR=#000000][B]>[/B][/COLOR][/COLOR]        
[COLOR=#009900][COLOR=#000000][B]<ip[/B][/COLOR] [COLOR=#000066]address[/COLOR]=[COLOR=#FF0000]"192.168.7.180"[/COLOR][COLOR=#000000][B]/>[/B][/COLOR][/COLOR]    
[COLOR=#009900][COLOR=#000000][B]</service> [/B][/COLOR][/COLOR][COLOR=#000000][B]</rm[/B][/COLOR][COLOR=#000000][B]>
[/B][/COLOR]
Cant figure out abut this IP - it belongs to?!?!
 

adamb

Renowned Member
Mar 1, 2012
1,163
44
68
Ok, tested my settup with:
Code:
fence_ipmilan -l User -p Passwd -P -a 10.10.10.6 -T 4 -o off -v
Server goes down. With:
Code:
fence_ipmilan -l User -p Passwd -P -a 10.10.10.6 -T 4 -o on -v
Server comes alive :)
My cluster.conf.new looks like this, but it doesnt go through validation :(
Code:
<?xml version="1.0"?><cluster name="cluster" config_version="4">


  <cman keyfile="/var/lib/pve-cluster/corosync.authkey">
  </cman>
  <fencedevices>
        <fencedevice agent="fence_ipmilan" name="ipmi1" lanplus="1" ipaddr="10.10.10.4" login="User" passwd="Passwd" power_wait="5"/>
        <fencedevice agent="fence_ipmilan" name="ipmi2" lanplus="1" ipaddr="10.10.10.5" login="User" passwd="Passwd" power_wait="5"/>
        <fencedevice agent="fence_ipmilan" name="ipmi3" lanplus="1" ipaddr="10.10.10.6" login="User" passwd="Passwd" power_wait="5"/>
  </fencedevices>
  <clusternodes>
  <clusternode name="vm1" votes="1" nodeid="1"/>
        <fence>
            <method name="1">
                 <device name="ipmi1"/>
            </method>
        </fence>
  <clusternode name="vm2" votes="1" nodeid="2"/>
        <fence>
            <method name="1">
                 <device name="ipmi2"/>
            </method>
        </fence>
  <clusternode name="vm3" votes="1" nodeid="3"/>
         <fence>
            <method name="1">
                 <device name="ipmi3"/>
            </method>
        </fence>
  </clusternodes>


</cluster>

Couldn't find out what does these lines are for in IPMI example:
Code:
[COLOR=#000000][B]<rm[/B][/COLOR][COLOR=#000000][B]>[/B][/COLOR]
    [COLOR=#009900][COLOR=#000000][B]<service[/B][/COLOR] [COLOR=#000066]autostart[/COLOR]=[COLOR=#FF0000]"1"[/COLOR] [COLOR=#000066]exclusive[/COLOR]=[COLOR=#FF0000]"0"[/COLOR] [COLOR=#000066]name[/COLOR]=[COLOR=#FF0000]"ha_test_ip"[/COLOR] [COLOR=#000066]recovery[/COLOR]=[COLOR=#FF0000]"relocate"[/COLOR][COLOR=#000000][B]>[/B][/COLOR][/COLOR]        
[COLOR=#009900][COLOR=#000000][B]<ip[/B][/COLOR] [COLOR=#000066]address[/COLOR]=[COLOR=#FF0000]"192.168.7.180"[/COLOR][COLOR=#000000][B]/>[/B][/COLOR][/COLOR]    
[COLOR=#009900][COLOR=#000000][B]</service> [/B][/COLOR][/COLOR][COLOR=#000000][B]</rm[/B][/COLOR][COLOR=#000000][B]>
[/B][/COLOR]
Cant figure out abut this IP - it belongs to?!?!

Here is an example of my config.

<?xml version="1.0"?>
<cluster config_version="29" name="fiosprox">
<cman expected_votes="3" keyfile="/var/lib/pve-cluster/corosync.authkey"/>
<quorumd allow_kill="0" interval="3" label="fiosprox_qdisk" master_wins="1" tko="10"/>
<totem token="54000"/>
<fencedevices>
<fencedevice agent="fence_ipmilan" ipaddr="10.80.12.129" lanplus="1" login="USERID" name="ipmi1" passwd="PASSW0RD" power_wait="5"/>
<fencedevice agent="fence_ipmilan" ipaddr="10.80.12.132" lanplus="1" login="USERID" name="ipmi2" passwd="PASSW0RD" power_wait="5"/>
</fencedevices>
<clusternodes>
<clusternode name="fiosprox1" nodeid="1" votes="1">
<fence>
<method name="1">
<device name="ipmi1"/>
</method>
</fence>
</clusternode>
<clusternode name="fiosprox2" nodeid="2" votes="1">
<fence>
<method name="1">
<device name="ipmi2"/>
</method>
</fence>
</clusternode>
</clusternodes>
<rm>
<pvevm autostart="1" vmid="102"/>
<pvevm autostart="1" vmid="100"/>
<pvevm autostart="1" vmid="101"/>
<pvevm autostart="1" vmid="104"/>
</rm>
</cluster>

Use this to check your config.


ccs_config_validate -v -f /etc/pve/cluster.conf.new
 

snpz

Member
Mar 18, 2013
20
0
21
Figured it out yesterday by myself :)
The problem was, that there was no closing tag</clusternode> after defining each node.
It works now!
Thanks a lot, adamb!!! :)
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!