HA

frantek

Renowned Member
May 30, 2009
171
7
83
Hi,

I've a HA test setup with NFS storage and 3 nodes. Everyting works as expected. VMs could be migrated from one node to an other. Fencing is set up as described in the wiki. But when I remove power from a node to simulate a failure the VMs running on this node are not automatically started on an other node. Why?

TIA
 
post your cluster.conf file

> cat /etc/pve/cluster.conf

and check logs.
 
cat /etc/cluster/cluster.conf
Code:
<?xml version="1.0"?>
<cluster config_version="9" name="CL-SSB">
  <cman keyfile="/var/lib/pve-cluster/corosync.authkey"/>
  <fencedevices>
    <fencedevice agent="fence_bladecenter" ipaddr="172.27.1.250" login="test" name="blade_mmc" passwd="test1"/>
  </fencedevices>
  <clusternodes>
    <clusternode name="sbapm01" nodeid="1" votes="1">
      <fence>
        <method name="1">
          <device port="1" name="blade_mmc"/>
        </method>
      </fence>
    </clusternode>
    <clusternode name="sbapm02" nodeid="2" votes="1">
      <fence>
        <method name="1">
          <device port="2" name="blade_mmc"/>
        </method>
      </fence>
    </clusternode>
    <clusternode name="sbapm03" nodeid="3" votes="1">
      <fence>
        <method name="1">
          <device port="3" name="blade_mmc"/>
        </method>
      </fence>
    </clusternode>
  </clusternodes>
  <rm>
    <pvevm autostart="1" vmid="100"/>
  </rm>
</cluster>

fence_tool ls
Code:
fence domain
member count  3
victim count  0
victim now    0
master nodeid 2
wait state    none
members       1 2 3

pvecm status
Code:
Version: 6.2.0
Config Version: 9
Cluster Name: CL-SSB
Cluster Id: 4284
Cluster Member: Yes
Cluster Generation: 88
Membership state: Cluster-Member
Nodes: 3
Expected votes: 3
Total votes: 3
Node votes: 1
Quorum: 2  
Active subsystems: 6
Flags: 
Ports Bound: 0  
Node name: sbapm01
Node ID: 1
Multicast addresses: 239.192.16.204 
Node addresses: 10.1.6.51

fence_bladecenter -h
Code:
Usage:
    fence_bladecenter [options]
Options:
   -a, --ip=[ip]                  IP address or hostname of fencing device
   -l, --username=[name]          Login name
   -p, --password=[password]      Login password or passphrase
   -c, --command-prompt=[prompt]  Force command prompt
   -x, --ssh                      Use ssh connection
   -n, --plug=[id]                Physical plug number on device, UUID or
                                        identification of machine
   -u, --ipport=[port]            TCP/UDP port to use
                                        (default 23, 22 if --ssh option is used)
   -4, --inet4-only               Forces agent to use IPv4 addresses only
   -6, --inet6-only               Forces agent to use IPv6 addresses only
   -S, --password-script=[script] Script to run to retrieve password
   -k, --identity-file=[filename] Identity file (private key) for ssh 
   --ssh-options=[options]      SSH options to use
   -o, --action=[action]          Action: status, reboot (default), off or on
   -v, --verbose                  Verbose mode
   -D, --debug-file=[debugfile]   Debugging to output file
   -V, --version                  Output version information and exit
   -h, --help                     Display this help and exit
   -C, --separator=[char]         Separator for CSV created by 'list' operation
   --missing-as-off               Missing port returns OFF instead of failure
   --power-timeout=[seconds]      Test X seconds for status change after ON/OFF
   --shell-timeout=[seconds]      Wait X seconds for cmd prompt after issuing command
   --login-timeout=[seconds]      Wait X seconds for cmd prompt after login
   --power-wait=[seconds]         Wait X seconds after issuing ON/OFF
   --delay=[seconds]              Wait X seconds before fencing is started
   --retry-on=[attempts]          Count of attempts to retry power on

fence_bladecenter -a 172.27.1.250 --username="test" --password="test1" -o status -n 2 -v
Code:
telnet> set binary
Negotiating binary mode with remote host.
telnet> open 172.27.1.250 -23
Trying 172.27.1.250...
Connected to 172.27.1.250.
Escape character is '^]'.

username: password: 

Hostname:              Blade4
Static IP address:     172.27.1.250
Burned-in MAC address: 00:11:25:C3:44:D0
DHCP:                  Disabled - Use static IP configuration.
Last login: Unable to connect/login to fencing device

fence_bladecenter -a 172.27.1.250 --username="test" --password="test1" -o status -n 2 -v -x
Code:
password:

Hostname:              Blade4
Static IP address:     172.27.1.250
Burned-in MAC address: 00:11:25:C3:44:D0
DHCP:                  Disabled - Use static IP configuration.
Last login: Tuesday November 5 2013 11:39 from 10.1.6.52 (Telnet)

system> env -T system:blade[2]
OK
system:blade[2]> power -state
On
system:blade[2]> env -T system
OK
system>Status: ON
 
Finaly found some time to assemble data:

rgmanager.log
Code:
Oct 28 08:31:33 rgmanager I am node #1
Oct 28 08:31:33 rgmanager Resource Group Manager Starting
Oct 28 08:31:33 rgmanager Loading Service Data
Oct 28 08:31:35 rgmanager Initializing Services
Oct 28 08:31:36 rgmanager [pvevm] VM 100 is already stopped
Oct 28 08:31:36 rgmanager Services Initialized
Oct 28 08:31:36 rgmanager State change: Local UP
Oct 28 08:31:36 rgmanager State change: sbapm02 UP
Oct 28 08:31:36 rgmanager State change: sbapm03 UP
Oct 28 08:31:37 rgmanager Migration: pvevm:100 is running on 3
Oct 28 08:32:48 rgmanager [pvevm] VM 100 is not running
Oct 28 08:33:03 rgmanager [pvevm] VM 100 is not running
Oct 28 08:33:05 rgmanager [pvevm] VM 100 is not running
Oct 28 08:34:32 rgmanager State change: sbapm03 DOWN
Oct 28 08:37:46 rgmanager Shutting down
Oct 28 08:37:56 rgmanager Shutting down
Oct 28 08:40:05 rgmanager Disconnecting from CMAN
Oct 28 08:40:05 rgmanager Exiting
Oct 28 08:43:45 rgmanager Waiting for quorum to form
Oct 28 08:44:23 rgmanager Quorum formed

fenced.log
Code:
Oct 18 12:54:51 fenced fenced 1364188437 started
Oct 28 08:34:32 fenced fencing node sbapm03
Oct 28 08:34:32 fenced fence sbapm03 dev 0.0 agent none result: error no method
Oct 28 08:34:32 fenced fence sbapm03 failed
Oct 28 08:34:35 fenced fencing node sbapm03
Oct 28 08:34:35 fenced fence sbapm03 dev 0.0 agent none result: error no method
Oct 28 08:34:35 fenced fence sbapm03 failed
Oct 28 08:34:38 fenced fencing node sbapm03
Oct 28 08:34:38 fenced fence sbapm03 dev 0.0 agent none result: error no method
Oct 28 08:34:38 fenced fence sbapm03 failed
Oct 28 09:34:40 fenced fenced 1364188437 started
Nov 05 11:43:52 fenced fenced 1364188437 started
dlm_controld.log
Code:
Oct 18 12:54:51 dlm_controld dlm_controld 1364188437 started
Oct 28 09:34:40 dlm_controld dlm_controld 1364188437 started
Nov 05 11:43:52 dlm_controld dlm_controld 1364188437 started
cpglockd.log
Code:
Oct 28 08:37:30 cpglockd cpglockd entering normal operation
Oct 28 08:40:08 cpglockd cman requested shutdown. Exiting.
 
corosync.log
Code:
Oct 18 12:54:47 corosync [MAIN  ] Corosync Cluster Engine ('1.4.5'): started and ready to provide service.
Oct 18 12:54:47 corosync [MAIN  ] Corosync built-in features: nss
Oct 18 12:54:47 corosync [MAIN  ] Successfully read config from /etc/cluster/cluster.conf
Oct 18 12:54:47 corosync [MAIN  ] Successfully parsed cman config
Oct 18 12:54:47 corosync [MAIN  ] Successfully configured openais services to load
Oct 18 12:54:47 corosync [TOTEM ] Initializing transport (UDP/IP Multicast).
Oct 18 12:54:47 corosync [TOTEM ] Initializing transmit/receive security: libtomcrypt SOBER128/SHA1HMAC (mode 0).
Oct 18 12:54:47 corosync [TOTEM ] The network interface [10.1.6.51] is now up.
Oct 18 12:54:47 corosync [QUORUM] Using quorum provider quorum_cman
Oct 18 12:54:47 corosync [SERV  ] Service engine loaded: corosync cluster quorum service v0.1
Oct 18 12:54:47 corosync [CMAN  ] CMAN 1364188437 (built Mar 25 2013 06:14:01) started
Oct 18 12:54:47 corosync [SERV  ] Service engine loaded: corosync CMAN membership service 2.90
Oct 18 12:54:47 corosync [SERV  ] Service engine loaded: openais cluster membership service B.01.01
Oct 18 12:54:47 corosync [SERV  ] Service engine loaded: openais event service B.01.01
Oct 18 12:54:47 corosync [SERV  ] Service engine loaded: openais checkpoint service B.01.01
Oct 18 12:54:47 corosync [SERV  ] Service engine loaded: openais message service B.03.01
Oct 18 12:54:47 corosync [SERV  ] Service engine loaded: openais distributed locking service B.03.01
Oct 18 12:54:47 corosync [SERV  ] Service engine loaded: openais timer service A.01.01
Oct 18 12:54:47 corosync [SERV  ] Service engine loaded: corosync extended virtual synchrony service
Oct 18 12:54:47 corosync [SERV  ] Service engine loaded: corosync configuration service
Oct 18 12:54:47 corosync [SERV  ] Service engine loaded: corosync cluster closed process group service v1.01
Oct 18 12:54:47 corosync [SERV  ] Service engine loaded: corosync cluster config database access v1.01
Oct 18 12:54:47 corosync [SERV  ] Service engine loaded: corosync profile loading service
Oct 18 12:54:47 corosync [QUORUM] Using quorum provider quorum_cman
Oct 18 12:54:47 corosync [SERV  ] Service engine loaded: corosync cluster quorum service v0.1
Oct 18 12:54:47 corosync [MAIN  ] Compatibility mode set to whitetank.  Using V1 and V2 of the synchronization engine.
Oct 18 12:54:47 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 18 12:54:47 corosync [CLM   ] New Configuration:
Oct 18 12:54:47 corosync [CLM   ] Members Left:
Oct 18 12:54:47 corosync [CLM   ] Members Joined:
Oct 18 12:54:47 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 18 12:54:47 corosync [CLM   ] New Configuration:
Oct 18 12:54:47 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 18 12:54:47 corosync [CLM   ] Members Left:
Oct 18 12:54:47 corosync [CLM   ] Members Joined:
Oct 18 12:54:47 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 18 12:54:47 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed.
Oct 18 12:54:47 corosync [CMAN  ] quorum regained, resuming activity
Oct 18 12:54:47 corosync [QUORUM] This node is within the primary component and will provide service.
Oct 18 12:54:47 corosync [QUORUM] Members[1]: 1
Oct 18 12:54:47 corosync [QUORUM] Members[1]: 1
Oct 18 12:54:47 corosync [CPG   ] chosen downlist: sender r(0) ip(10.1.6.51) ; members(old:0 left:0)
Oct 18 12:54:47 corosync [MAIN  ] Completed service synchronization, ready to provide service.
Oct 18 12:58:03 corosync [QUORUM] Members[1]: 1
Oct 18 12:58:17 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 18 12:58:17 corosync [CLM   ] New Configuration:
Oct 18 12:58:17 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 18 12:58:17 corosync [CLM   ] Members Left:
Oct 18 12:58:17 corosync [CLM   ] Members Joined:
Oct 18 12:58:17 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 18 12:58:17 corosync [CLM   ] New Configuration:
Oct 18 12:58:17 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 18 12:58:17 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 18 12:58:17 corosync [CLM   ] Members Left:
Oct 18 12:58:17 corosync [CLM   ] Members Joined:
Oct 18 12:58:17 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 18 12:58:17 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed.
Oct 18 12:58:17 corosync [QUORUM] Members[2]: 1 2
Oct 18 12:58:17 corosync [QUORUM] Members[2]: 1 2
Oct 18 12:58:17 corosync [QUORUM] Members[2]: 1 2
Oct 18 12:58:17 corosync [CPG   ] chosen downlist: sender r(0) ip(10.1.6.51) ; members(old:1 left:0)
Oct 18 12:58:17 corosync [MAIN  ] Completed service synchronization, ready to provide service.
Oct 18 11:26:32 corosync [QUORUM] Members[2]: 1 2
Oct 18 11:26:46 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 18 11:26:46 corosync [CLM   ] New Configuration:
Oct 18 11:26:46 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 18 11:26:46 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 18 11:26:46 corosync [CLM   ] Members Left:
Oct 18 11:26:46 corosync [CLM   ] Members Joined:
Oct 18 11:26:46 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 18 11:26:46 corosync [CLM   ] New Configuration:
Oct 18 11:26:46 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 18 11:26:46 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 18 11:26:46 corosync [CLM   ]     r(0) ip(10.1.6.53) 
Oct 18 11:26:46 corosync [CLM   ] Members Left:
Oct 18 11:26:46 corosync [CLM   ] Members Joined:
Oct 18 11:26:46 corosync [CLM   ]     r(0) ip(10.1.6.53) 
Oct 18 11:26:46 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed.
Oct 18 11:26:46 corosync [QUORUM] Members[3]: 1 2 3
Oct 18 11:26:46 corosync [QUORUM] Members[3]: 1 2 3
Oct 18 11:26:46 corosync [QUORUM] Members[3]: 1 2 3
Oct 18 11:26:46 corosync [CPG   ] chosen downlist: sender r(0) ip(10.1.6.51) ; members(old:2 left:0)
Oct 18 11:26:46 corosync [MAIN  ] Completed service synchronization, ready to provide service.
Oct 18 11:28:47 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 18 11:28:47 corosync [CLM   ] New Configuration:
Oct 18 11:28:47 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 18 11:28:47 corosync [CLM   ]     r(0) ip(10.1.6.53) 
Oct 18 11:28:47 corosync [CLM   ] Members Left:
Oct 18 11:28:47 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 18 11:28:47 corosync [CLM   ] Members Joined:
Oct 18 11:28:47 corosync [QUORUM] Members[2]: 1 3
Oct 18 11:28:47 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 18 11:28:47 corosync [CLM   ] New Configuration:
Oct 18 11:28:47 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 18 11:28:47 corosync [CLM   ]     r(0) ip(10.1.6.53) 
Oct 18 11:28:47 corosync [CLM   ] Members Left:
Oct 18 11:28:47 corosync [CLM   ] Members Joined:
Oct 18 11:28:47 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed.
Oct 18 11:28:47 corosync [QUORUM] Members[2]: 1 3
Oct 18 11:28:47 corosync [CPG   ] chosen downlist: sender r(0) ip(10.1.6.51) ; members(old:3 left:1)
Oct 18 11:28:47 corosync [MAIN  ] Completed service synchronization, ready to provide service.
Oct 18 11:38:59 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 18 11:38:59 corosync [CLM   ] New Configuration:
Oct 18 11:38:59 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 18 11:38:59 corosync [CLM   ]     r(0) ip(10.1.6.53) 
Oct 18 11:38:59 corosync [CLM   ] Members Left:
Oct 18 11:38:59 corosync [CLM   ] Members Joined:
Oct 18 11:38:59 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 18 11:38:59 corosync [CLM   ] New Configuration:
Oct 18 11:38:59 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 18 11:38:59 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 18 11:38:59 corosync [CLM   ]     r(0) ip(10.1.6.53) 
Oct 18 11:38:59 corosync [CLM   ] Members Left:
Oct 18 11:38:59 corosync [CLM   ] Members Joined:
Oct 18 11:38:59 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 18 11:38:59 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed.
Oct 18 11:38:59 corosync [QUORUM] Members[3]: 1 2 3
Oct 18 11:38:59 corosync [QUORUM] Members[3]: 1 2 3
Oct 18 11:38:59 corosync [QUORUM] Members[3]: 1 2 3
Oct 18 11:38:59 corosync [CPG   ] chosen downlist: sender r(0) ip(10.1.6.51) ; members(old:2 left:0)
Oct 18 11:38:59 corosync [MAIN  ] Completed service synchronization, ready to provide service.
Oct 22 10:27:29 corosync [TOTEM ] A processor failed, forming new configuration.
Oct 22 10:27:31 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 22 10:27:31 corosync [CLM   ] New Configuration:
Oct 22 10:27:31 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 22 10:27:31 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 22 10:27:31 corosync [CLM   ] Members Left:
Oct 22 10:27:31 corosync [CLM   ]     r(0) ip(10.1.6.53) 
Oct 22 10:27:31 corosync [CLM   ] Members Joined:
Oct 22 10:27:31 corosync [QUORUM] Members[2]: 1 2
Oct 22 10:27:31 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 22 10:27:31 corosync [CLM   ] New Configuration:
Oct 22 10:27:31 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 22 10:27:31 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 22 10:27:31 corosync [CLM   ] Members Left:
Oct 22 10:27:31 corosync [CLM   ] Members Joined:
Oct 22 10:27:31 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed.
Oct 22 10:27:31 corosync [QUORUM] Members[2]: 1 2
Oct 22 10:27:31 corosync [CPG   ] chosen downlist: sender r(0) ip(10.1.6.51) ; members(old:3 left:1)
Oct 22 10:27:31 corosync [MAIN  ] Completed service synchronization, ready to provide service.
Oct 22 10:34:23 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 22 10:34:23 corosync [CLM   ] New Configuration:
Oct 22 10:34:23 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 22 10:34:23 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 22 10:34:23 corosync [CLM   ] Members Left:
Oct 22 10:34:23 corosync [CLM   ] Members Joined:
Oct 22 10:34:23 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 22 10:34:23 corosync [CLM   ] New Configuration:
Oct 22 10:34:23 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 22 10:34:23 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 22 10:34:23 corosync [CLM   ] Members Left:
Oct 22 10:34:23 corosync [CLM   ] Members Joined:
Oct 22 10:34:23 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed.
Oct 22 10:34:23 corosync [QUORUM] Members[2]: 1 2
Oct 22 10:34:23 corosync [CPG   ] chosen downlist: sender r(0) ip(10.1.6.51) ; members(old:2 left:0)
Oct 22 10:34:23 corosync [MAIN  ] Completed service synchronization, ready to provide service.
Oct 22 10:34:25 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 22 10:34:25 corosync [CLM   ] New Configuration:
Oct 22 10:34:25 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 22 10:34:25 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 22 10:34:25 corosync [CLM   ] Members Left:
Oct 22 10:34:25 corosync [CLM   ] Members Joined:
Oct 22 10:34:25 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 22 10:34:25 corosync [CLM   ] New Configuration:
Oct 22 10:34:25 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 22 10:34:25 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 22 10:34:25 corosync [CLM   ]     r(0) ip(10.1.6.53) 
Oct 22 10:34:25 corosync [CLM   ] Members Left:
Oct 22 10:34:25 corosync [CLM   ] Members Joined:
Oct 22 10:34:25 corosync [CLM   ]     r(0) ip(10.1.6.53) 
Oct 22 10:34:25 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed.
Oct 22 10:34:25 corosync [QUORUM] Members[3]: 1 2 3
Oct 22 10:34:25 corosync [QUORUM] Members[3]: 1 2 3
Oct 22 10:34:25 corosync [QUORUM] Members[3]: 1 2 3
Oct 22 10:34:25 corosync [CPG   ] chosen downlist: sender r(0) ip(10.1.6.51) ; members(old:2 left:0)
Oct 22 10:34:25 corosync [MAIN  ] Completed service synchronization, ready to provide service.
Oct 28 08:13:40 corosync [QUORUM] Members[3]: 1 2 3
Oct 28 08:18:57 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 28 08:18:57 corosync [CLM   ] New Configuration:
Oct 28 08:18:57 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 28 08:18:57 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 28 08:18:57 corosync [CLM   ] Members Left:
Oct 28 08:18:57 corosync [CLM   ]     r(0) ip(10.1.6.53) 
Oct 28 08:18:57 corosync [CLM   ] Members Joined:
Oct 28 08:18:57 corosync [QUORUM] Members[2]: 1 2
Oct 28 08:18:57 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 28 08:18:57 corosync [CLM   ] New Configuration:
Oct 28 08:18:57 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 28 08:18:57 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 28 08:18:57 corosync [CLM   ] Members Left:
Oct 28 08:18:57 corosync [CLM   ] Members Joined:
Oct 28 08:18:57 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed.
Oct 28 08:18:57 corosync [QUORUM] Members[2]: 1 2
Oct 28 08:18:57 corosync [CPG   ] chosen downlist: sender r(0) ip(10.1.6.51) ; members(old:3 left:1)
Oct 28 08:18:57 corosync [MAIN  ] Completed service synchronization, ready to provide service.
Oct 28 08:22:39 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 28 08:22:39 corosync [CLM   ] New Configuration:
Oct 28 08:22:39 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 28 08:22:39 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 28 08:22:39 corosync [CLM   ] Members Left:
Oct 28 08:22:39 corosync [CLM   ] Members Joined:
Oct 28 08:22:39 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 28 08:22:39 corosync [CLM   ] New Configuration:
Oct 28 08:22:39 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 28 08:22:39 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 28 08:22:39 corosync [CLM   ]     r(0) ip(10.1.6.53) 
Oct 28 08:22:39 corosync [CLM   ] Members Left:
Oct 28 08:22:39 corosync [CLM   ] Members Joined:
Oct 28 08:22:39 corosync [CLM   ]     r(0) ip(10.1.6.53) 
Oct 28 08:22:39 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed.
Oct 28 08:22:39 corosync [QUORUM] Members[3]: 1 2 3
Oct 28 08:22:39 corosync [QUORUM] Members[3]: 1 2 3
Oct 28 08:22:39 corosync [QUORUM] Members[3]: 1 2 3
Oct 28 08:22:39 corosync [CPG   ] chosen downlist: sender r(0) ip(10.1.6.51) ; members(old:2 left:0)
Oct 28 08:22:39 corosync [MAIN  ] Completed service synchronization, ready to provide service.
Oct 28 08:34:30 corosync [TOTEM ] A processor failed, forming new configuration.
Oct 28 08:34:32 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 28 08:34:32 corosync [CLM   ] New Configuration:
Oct 28 08:34:32 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 28 08:34:32 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 28 08:34:32 corosync [CLM   ] Members Left:
Oct 28 08:34:32 corosync [CLM   ]     r(0) ip(10.1.6.53) 
Oct 28 08:34:32 corosync [CLM   ] Members Joined:
Oct 28 08:34:32 corosync [QUORUM] Members[2]: 1 2
Oct 28 08:34:32 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 28 08:34:32 corosync [CLM   ] New Configuration:
Oct 28 08:34:32 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 28 08:34:32 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 28 08:34:32 corosync [CLM   ] Members Left:
Oct 28 08:34:32 corosync [CLM   ] Members Joined:
Oct 28 08:34:32 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed.
Oct 28 08:34:32 corosync [QUORUM] Members[2]: 1 2
Oct 28 08:34:32 corosync [CPG   ] chosen downlist: sender r(0) ip(10.1.6.51) ; members(old:3 left:1)
Oct 28 08:34:32 corosync [MAIN  ] Completed service synchronization, ready to provide service.
Oct 28 08:40:01 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 28 08:40:01 corosync [CLM   ] New Configuration:
Oct 28 08:40:01 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 28 08:40:01 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 28 08:40:01 corosync [CLM   ] Members Left:
Oct 28 08:40:01 corosync [CLM   ] Members Joined:
Oct 28 08:40:01 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 28 08:40:01 corosync [CLM   ] New Configuration:
Oct 28 08:40:01 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 28 08:40:01 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 28 08:40:01 corosync [CLM   ]     r(0) ip(10.1.6.53) 
Oct 28 08:40:01 corosync [CLM   ] Members Left:
Oct 28 08:40:01 corosync [CLM   ] Members Joined:
Oct 28 08:40:01 corosync [CLM   ]     r(0) ip(10.1.6.53) 
Oct 28 08:40:01 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed.
Oct 28 08:40:01 corosync [QUORUM] Members[3]: 1 2 3
Oct 28 08:40:01 corosync [QUORUM] Members[3]: 1 2 3
Oct 28 08:40:01 corosync [QUORUM] Members[3]: 1 2 3
Oct 28 08:40:01 corosync [CPG   ] chosen downlist: sender r(0) ip(10.1.6.51) ; members(old:2 left:0)
Oct 28 08:40:01 corosync [MAIN  ] Completed service synchronization, ready to provide service.
Oct 28 08:40:08 corosync [SERV  ] Unloading all Corosync service engines.
Oct 28 08:40:08 corosync [SERV  ] Service engine unloaded: corosync extended virtual synchrony service
Oct 28 08:40:08 corosync [SERV  ] Service engine unloaded: corosync configuration service
Oct 28 08:40:08 corosync [SERV  ] Service engine unloaded: corosync cluster closed process group service v1.01
Oct 28 08:40:08 corosync [SERV  ] Service engine unloaded: corosync cluster config database access v1.01
Oct 28 08:40:08 corosync [SERV  ] Service engine unloaded: corosync profile loading service
Oct 28 08:40:08 corosync [SERV  ] Service engine unloaded: openais cluster membership service B.01.01
Oct 28 08:40:08 corosync [SERV  ] Service engine unloaded: openais checkpoint service B.01.01
Oct 28 08:40:08 corosync [SERV  ] Service engine unloaded: openais event service B.01.01
Oct 28 08:40:08 corosync [SERV  ] Service engine unloaded: openais distributed locking service B.03.01
Oct 28 08:40:08 corosync [SERV  ] Service engine unloaded: openais message service B.03.01
Oct 28 08:40:08 corosync [SERV  ] Service engine unloaded: corosync CMAN membership service 2.90
Oct 28 08:40:08 corosync [SERV  ] Service engine unloaded: corosync cluster quorum service v0.1
Oct 28 08:40:08 corosync [SERV  ] Service engine unloaded: openais timer service A.01.01
Oct 28 08:40:08 corosync [MAIN  ] Corosync Cluster Engine exiting with status 0 at main.c:1893.
Oct 28 08:42:55 corosync [MAIN  ] Corosync Cluster Engine ('1.4.5'): started and ready to provide service.
Oct 28 08:42:55 corosync [MAIN  ] Corosync built-in features: nss
Oct 28 08:42:55 corosync [MAIN  ] Successfully read config from /etc/cluster/cluster.conf
Oct 28 08:42:55 corosync [MAIN  ] Successfully parsed cman config
Oct 28 08:42:55 corosync [MAIN  ] Successfully configured openais services to load
Oct 28 08:42:55 corosync [TOTEM ] Initializing transport (UDP/IP Multicast).
Oct 28 08:42:55 corosync [TOTEM ] Initializing transmit/receive security: libtomcrypt SOBER128/SHA1HMAC (mode 0).
Oct 28 08:42:55 corosync [TOTEM ] The network interface [10.1.6.51] is now up.
Oct 28 08:42:55 corosync [QUORUM] Using quorum provider quorum_cman
Oct 28 08:42:55 corosync [SERV  ] Service engine loaded: corosync cluster quorum service v0.1
Oct 28 08:42:55 corosync [CMAN  ] CMAN 1364188437 (built Mar 25 2013 06:14:01) started
Oct 28 08:42:55 corosync [SERV  ] Service engine loaded: corosync CMAN membership service 2.90
Oct 28 08:42:55 corosync [SERV  ] Service engine loaded: openais cluster membership service B.01.01
Oct 28 08:42:55 corosync [SERV  ] Service engine loaded: openais event service B.01.01
Oct 28 08:42:55 corosync [SERV  ] Service engine loaded: openais checkpoint service B.01.01
Oct 28 08:42:55 corosync [SERV  ] Service engine loaded: openais message service B.03.01
Oct 28 08:42:55 corosync [SERV  ] Service engine loaded: openais distributed locking service B.03.01
Oct 28 08:42:55 corosync [SERV  ] Service engine loaded: openais timer service A.01.01
Oct 28 08:42:55 corosync [SERV  ] Service engine loaded: corosync extended virtual synchrony service
Oct 28 08:42:55 corosync [SERV  ] Service engine loaded: corosync configuration service
Oct 28 08:42:55 corosync [SERV  ] Service engine loaded: corosync cluster closed process group service v1.01
Oct 28 08:42:55 corosync [SERV  ] Service engine loaded: corosync cluster config database access v1.01
Oct 28 08:42:55 corosync [SERV  ] Service engine loaded: corosync profile loading service
Oct 28 08:42:55 corosync [QUORUM] Using quorum provider quorum_cman
Oct 28 08:42:55 corosync [SERV  ] Service engine loaded: corosync cluster quorum service v0.1
Oct 28 08:42:55 corosync [MAIN  ] Compatibility mode set to whitetank.  Using V1 and V2 of the synchronization engine.
Oct 28 08:42:55 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 28 08:42:55 corosync [CLM   ] New Configuration:
Oct 28 08:42:55 corosync [CLM   ] Members Left:
Oct 28 08:42:55 corosync [CLM   ] Members Joined:
Oct 28 08:42:55 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 28 08:42:55 corosync [CLM   ] New Configuration:
Oct 28 08:42:55 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 28 08:42:55 corosync [CLM   ] Members Left:
Oct 28 08:42:55 corosync [CLM   ] Members Joined:
Oct 28 08:42:55 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 28 08:42:55 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed.
Oct 28 08:42:55 corosync [QUORUM] Members[1]: 1
Oct 28 08:42:55 corosync [QUORUM] Members[1]: 1
Oct 28 08:42:55 corosync [CPG   ] chosen downlist: sender r(0) ip(10.1.6.51) ; members(old:0 left:0)
Oct 28 08:42:55 corosync [MAIN  ] Completed service synchronization, ready to provide service.
Oct 28 08:44:22 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 28 08:44:22 corosync [CLM   ] New Configuration:
Oct 28 08:44:22 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 28 08:44:22 corosync [CLM   ] Members Left:
Oct 28 08:44:22 corosync [CLM   ] Members Joined:
Oct 28 08:44:22 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 28 08:44:22 corosync [CLM   ] New Configuration:
Oct 28 08:44:22 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 28 08:44:22 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 28 08:44:22 corosync [CLM   ] Members Left:
Oct 28 08:44:22 corosync [CLM   ] Members Joined:
Oct 28 08:44:22 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 28 08:44:22 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed.
Oct 28 08:44:22 corosync [CMAN  ] quorum regained, resuming activity
Oct 28 08:44:22 corosync [QUORUM] This node is within the primary component and will provide service.
Oct 28 08:44:22 corosync [QUORUM] Members[2]: 1 2
Oct 28 08:44:22 corosync [QUORUM] Members[2]: 1 2
Oct 28 08:44:22 corosync [CPG   ] chosen downlist: sender r(0) ip(10.1.6.51) ; members(old:1 left:0)
Oct 28 08:44:22 corosync [MAIN  ] Completed service synchronization, ready to provide service.
Oct 28 09:14:23 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 28 09:14:23 corosync [CLM   ] New Configuration:
Oct 28 09:14:23 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 28 09:14:23 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 28 09:14:23 corosync [CLM   ] Members Left:
Oct 28 09:14:23 corosync [CLM   ] Members Joined:
Oct 28 09:14:23 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 28 09:14:23 corosync [CLM   ] New Configuration:
Oct 28 09:14:23 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 28 09:14:23 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 28 09:14:23 corosync [CLM   ]     r(0) ip(10.1.6.53) 
Oct 28 09:14:23 corosync [CLM   ] Members Left:
Oct 28 09:14:23 corosync [CLM   ] Members Joined:
Oct 28 09:14:23 corosync [CLM   ]     r(0) ip(10.1.6.53) 
Oct 28 09:14:23 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed.
Oct 28 09:14:23 corosync [QUORUM] Members[3]: 1 2 3
Oct 28 09:14:23 corosync [QUORUM] Members[3]: 1 2 3
Oct 28 09:14:23 corosync [CPG   ] chosen downlist: sender r(0) ip(10.1.6.51) ; members(old:2 left:0)
Oct 28 09:14:23 corosync [MAIN  ] Completed service synchronization, ready to provide service.
Oct 28 09:16:25 corosync [TOTEM ] A processor failed, forming new configuration.
Oct 28 09:16:37 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 28 09:16:37 corosync [CLM   ] New Configuration:
Oct 28 09:16:37 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 28 09:16:37 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 28 09:16:37 corosync [CLM   ] Members Left:
Oct 28 09:16:37 corosync [CLM   ]     r(0) ip(10.1.6.53) 
Oct 28 09:16:37 corosync [CLM   ] Members Joined:
Oct 28 09:16:37 corosync [QUORUM] Members[2]: 1 2
Oct 28 09:16:37 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 28 09:16:37 corosync [CLM   ] New Configuration:
Oct 28 09:16:37 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 28 09:16:37 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 28 09:16:37 corosync [CLM   ] Members Left:
Oct 28 09:16:37 corosync [CLM   ] Members Joined:
Oct 28 09:16:37 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed.
Oct 28 09:16:37 corosync [CPG   ] chosen downlist: sender r(0) ip(10.1.6.51) ; members(old:3 left:1)
Oct 28 09:16:37 corosync [MAIN  ] Completed service synchronization, ready to provide service.
 
Code:
Oct 28 09:19:41 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 28 09:19:41 corosync [CLM   ] New Configuration:
Oct 28 09:19:41 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 28 09:19:41 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 28 09:19:41 corosync [CLM   ] Members Left:
Oct 28 09:19:41 corosync [CLM   ] Members Joined:
Oct 28 09:19:41 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 28 09:19:41 corosync [CLM   ] New Configuration:
Oct 28 09:19:41 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 28 09:19:41 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 28 09:19:41 corosync [CLM   ]     r(0) ip(10.1.6.53) 
Oct 28 09:19:41 corosync [CLM   ] Members Left:
Oct 28 09:19:41 corosync [CLM   ] Members Joined:
Oct 28 09:19:41 corosync [CLM   ]     r(0) ip(10.1.6.53) 
Oct 28 09:19:41 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed.
Oct 28 09:19:41 corosync [QUORUM] Members[3]: 1 2 3
Oct 28 09:19:41 corosync [QUORUM] Members[3]: 1 2 3
Oct 28 09:19:41 corosync [CPG   ] chosen downlist: sender r(0) ip(10.1.6.51) ; members(old:2 left:0)
Oct 28 09:19:41 corosync [MAIN  ] Completed service synchronization, ready to provide service.
Oct 28 09:20:55 corosync [TOTEM ] A processor failed, forming new configuration.
Oct 28 09:21:07 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 28 09:21:07 corosync [CLM   ] New Configuration:
Oct 28 09:21:07 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 28 09:21:07 corosync [CLM   ]     r(0) ip(10.1.6.53) 
Oct 28 09:21:07 corosync [CLM   ] Members Left:
Oct 28 09:21:07 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 28 09:21:07 corosync [CLM   ] Members Joined:
Oct 28 09:21:07 corosync [QUORUM] Members[2]: 1 3
Oct 28 09:21:07 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 28 09:21:07 corosync [CLM   ] New Configuration:
Oct 28 09:21:07 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 28 09:21:07 corosync [CLM   ]     r(0) ip(10.1.6.53) 
Oct 28 09:21:07 corosync [CLM   ] Members Left:
Oct 28 09:21:07 corosync [CLM   ] Members Joined:
Oct 28 09:21:07 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed.
Oct 28 09:21:07 corosync [CPG   ] chosen downlist: sender r(0) ip(10.1.6.51) ; members(old:3 left:1)
Oct 28 09:21:07 corosync [MAIN  ] Completed service synchronization, ready to provide service.
Oct 28 09:28:23 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 28 09:28:23 corosync [CLM   ] New Configuration:
Oct 28 09:28:23 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 28 09:28:23 corosync [CLM   ]     r(0) ip(10.1.6.53) 
Oct 28 09:28:23 corosync [CLM   ] Members Left:
Oct 28 09:28:23 corosync [CLM   ] Members Joined:
Oct 28 09:28:23 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 28 09:28:23 corosync [CLM   ] New Configuration:
Oct 28 09:28:23 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 28 09:28:23 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 28 09:28:23 corosync [CLM   ]     r(0) ip(10.1.6.53) 
Oct 28 09:28:23 corosync [CLM   ] Members Left:
Oct 28 09:28:23 corosync [CLM   ] Members Joined:
Oct 28 09:28:23 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 28 09:28:23 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed.
Oct 28 09:28:23 corosync [QUORUM] Members[3]: 1 2 3
Oct 28 09:28:23 corosync [QUORUM] Members[3]: 1 2 3
Oct 28 09:28:23 corosync [CPG   ] chosen downlist: sender r(0) ip(10.1.6.51) ; members(old:2 left:0)
Oct 28 09:28:23 corosync [MAIN  ] Completed service synchronization, ready to provide service.
Oct 28 09:31:31 corosync [SERV  ] Unloading all Corosync service engines.
Oct 28 09:31:31 corosync [SERV  ] Service engine unloaded: corosync extended virtual synchrony service
Oct 28 09:31:31 corosync [SERV  ] Service engine unloaded: corosync configuration service
Oct 28 09:31:31 corosync [SERV  ] Service engine unloaded: corosync cluster closed process group service v1.01
Oct 28 09:31:31 corosync [SERV  ] Service engine unloaded: corosync cluster config database access v1.01
Oct 28 09:31:31 corosync [SERV  ] Service engine unloaded: corosync profile loading service
Oct 28 09:31:31 corosync [SERV  ] Service engine unloaded: openais cluster membership service B.01.01
Oct 28 09:31:31 corosync [SERV  ] Service engine unloaded: openais checkpoint service B.01.01
Oct 28 09:31:31 corosync [SERV  ] Service engine unloaded: openais event service B.01.01
Oct 28 09:31:31 corosync [SERV  ] Service engine unloaded: openais distributed locking service B.03.01
Oct 28 09:31:31 corosync [SERV  ] Service engine unloaded: openais message service B.03.01
Oct 28 09:31:31 corosync [SERV  ] Service engine unloaded: corosync CMAN membership service 2.90
Oct 28 09:31:31 corosync [SERV  ] Service engine unloaded: corosync cluster quorum service v0.1
Oct 28 09:31:31 corosync [SERV  ] Service engine unloaded: openais timer service A.01.01
Oct 28 09:31:31 corosync [MAIN  ] Corosync Cluster Engine exiting with status 0 at main.c:1893.
Oct 28 09:34:17 corosync [MAIN  ] Corosync Cluster Engine ('1.4.5'): started and ready to provide service.
Oct 28 09:34:17 corosync [MAIN  ] Corosync built-in features: nss
Oct 28 09:34:17 corosync [MAIN  ] Successfully read config from /etc/cluster/cluster.conf
Oct 28 09:34:17 corosync [MAIN  ] Successfully parsed cman config
Oct 28 09:34:17 corosync [MAIN  ] Successfully configured openais services to load
Oct 28 09:34:17 corosync [TOTEM ] Initializing transport (UDP/IP Multicast).
Oct 28 09:34:17 corosync [TOTEM ] Initializing transmit/receive security: libtomcrypt SOBER128/SHA1HMAC (mode 0).
Oct 28 09:34:17 corosync [TOTEM ] The network interface [10.1.6.51] is now up.
Oct 28 09:34:17 corosync [QUORUM] Using quorum provider quorum_cman
Oct 28 09:34:17 corosync [SERV  ] Service engine loaded: corosync cluster quorum service v0.1
Oct 28 09:34:17 corosync [CMAN  ] CMAN 1364188437 (built Mar 25 2013 06:14:01) started
Oct 28 09:34:17 corosync [SERV  ] Service engine loaded: corosync CMAN membership service 2.90
Oct 28 09:34:17 corosync [SERV  ] Service engine loaded: openais cluster membership service B.01.01
Oct 28 09:34:17 corosync [SERV  ] Service engine loaded: openais event service B.01.01
Oct 28 09:34:17 corosync [SERV  ] Service engine loaded: openais checkpoint service B.01.01
Oct 28 09:34:17 corosync [SERV  ] Service engine loaded: openais message service B.03.01
Oct 28 09:34:17 corosync [SERV  ] Service engine loaded: openais distributed locking service B.03.01
Oct 28 09:34:17 corosync [SERV  ] Service engine loaded: openais timer service A.01.01
Oct 28 09:34:17 corosync [SERV  ] Service engine loaded: corosync extended virtual synchrony service
Oct 28 09:34:17 corosync [SERV  ] Service engine loaded: corosync configuration service
Oct 28 09:34:17 corosync [SERV  ] Service engine loaded: corosync cluster closed process group service v1.01
Oct 28 09:34:17 corosync [SERV  ] Service engine loaded: corosync cluster config database access v1.01
Oct 28 09:34:17 corosync [SERV  ] Service engine loaded: corosync profile loading service
Oct 28 09:34:17 corosync [QUORUM] Using quorum provider quorum_cman
Oct 28 09:34:17 corosync [SERV  ] Service engine loaded: corosync cluster quorum service v0.1
Oct 28 09:34:17 corosync [MAIN  ] Compatibility mode set to whitetank.  Using V1 and V2 of the synchronization engine.
Oct 28 09:34:17 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 28 09:34:17 corosync [CLM   ] New Configuration:
Oct 28 09:34:17 corosync [CLM   ] Members Left:
Oct 28 09:34:17 corosync [CLM   ] Members Joined:
Oct 28 09:34:17 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 28 09:34:17 corosync [CLM   ] New Configuration:
Oct 28 09:34:17 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 28 09:34:17 corosync [CLM   ] Members Left:
Oct 28 09:34:17 corosync [CLM   ] Members Joined:
Oct 28 09:34:17 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 28 09:34:17 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed.
Oct 28 09:34:17 corosync [QUORUM] Members[1]: 1
Oct 28 09:34:17 corosync [QUORUM] Members[1]: 1
Oct 28 09:34:17 corosync [CPG   ] chosen downlist: sender r(0) ip(10.1.6.51) ; members(old:0 left:0)
Oct 28 09:34:17 corosync [MAIN  ] Completed service synchronization, ready to provide service.
Oct 28 09:34:39 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 28 09:34:39 corosync [CLM   ] New Configuration:
Oct 28 09:34:39 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 28 09:34:39 corosync [CLM   ] Members Left:
Oct 28 09:34:39 corosync [CLM   ] Members Joined:
Oct 28 09:34:39 corosync [CLM   ] CLM CONFIGURATION CHANGE
Oct 28 09:34:39 corosync [CLM   ] New Configuration:
Oct 28 09:34:39 corosync [CLM   ]     r(0) ip(10.1.6.51) 
Oct 28 09:34:39 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 28 09:34:39 corosync [CLM   ]     r(0) ip(10.1.6.53) 
Oct 28 09:34:39 corosync [CLM   ] Members Left:
Oct 28 09:34:39 corosync [CLM   ] Members Joined:
Oct 28 09:34:39 corosync [CLM   ]     r(0) ip(10.1.6.52) 
Oct 28 09:34:39 corosync [CLM   ]     r(0) ip(10.1.6.53) 
Oct 28 09:34:39 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed.
Oct 28 09:34:39 corosync [CMAN  ] quorum regained, resuming activity
Oct 28 09:34:39 corosync [QUORUM] This node is within the primary component and will provide service.
Oct 28 09:34:39 corosync [QUORUM] Members[2]: 1 2
Oct 28 09:34:39 corosync [QUORUM] Members[2]: 1 2
Oct 28 09:34:39 corosync [QUORUM] Members[3]: 1 2 3
Oct 28 09:34:39 corosync [QUORUM] Members[3]: 1 2 3
Oct 28 09:34:39 corosync [CPG   ] chosen downlist: sender r(0) ip(10.1.6.52) ; members(old:2 left:0)
Oct 28 09:34:39 corosync [MAIN  ] Completed service synchronization, ready to provide service.
Oct 28 09:50:00 corosync [QUORUM] Members[3]: 1 2 3
Oct 28 10:24:12 corosync [QUORUM] Members[3]: 1 2 3
Oct 28 10:55:04 corosync [QUORUM] Members[3]: 1 2 3
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!