cluster lost . urgent *solved *

RobFantini

Famous Member
May 24, 2012
2,042
109
133
Boston,Mass
Hello
somehow we had network issues, and quorum was lost.

network is fixed.

but on all nodes in pve web page : only local node is green, the other 3 are read.

multicast test shows all ok.

all nodes are pveversion -v look like this:
Code:
proxmox-ve-2.6.32: 3.1-109 (running kernel: 2.6.32-23-pve)
pve-manager: 3.1-4 (running version: 3.1-4/f6816604)
pve-kernel-2.6.32-20-pve: 2.6.32-100
pve-kernel-2.6.32-22-pve: 2.6.32-107
pve-kernel-2.6.32-17-pve: 2.6.32-83
pve-kernel-2.6.32-11-pve: 2.6.32-66
pve-kernel-2.6.32-23-pve: 2.6.32-109
lvm2: 2.02.98-pve4
clvm: 2.02.98-pve4
corosync-pve: 1.4.5-1
openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.0-2
pve-cluster: 3.0-7
qemu-server: 3.1-1
pve-firmware: 1.0-23
libpve-common-perl: 3.0-6
libpve-access-control: 3.0-6
libpve-storage-perl: 3.0-10
pve-libspice-server1: 0.12.4-1
vncterm: 1.1-4
vzctl: 4.0-1pve3
vzprocps: 2.0.11-2
vzquota: 3.1-2
pve-qemu-kvm: 1.4-17
ksm-control-daemon: 1.1-1
glusterfs-client: 3.4.0-2


How can we get the cluster back?
 
Last edited:
Re: cluster lost . urgent

debugging data. more to be added soon
Code:
fbc87  /var/lib/vz/home/rob # pvecm nodes
Node  Sts   Inc   Joined               Name
   1   M  25112   2013-09-11 11:45:10  fbc3
   2   M  24104   2013-09-11 10:52:45  fbc87
   3   M  25108   2013-09-11 11:45:05  s035
   5   M  24964   2013-09-11 11:16:26  s012