[SOLVED] Problem with cluster. Corosync won't start.

[ookonek]

Hello,

I had cluster with two nodes and it was damaged. Now I can't make backup and others.
I know, I also can stop pve-cluster and run it in local mode (pmxcfs -l), make backup and re-install all, but maybe someone could help me...

root@srv1:/# pveversion -v
proxmox-ve: 4.1-26 (running kernel: 4.2.6-1-pve)
pve-manager: 4.1-1 (running version: 4.1-1/2f9650d4)
pve-kernel-4.2.6-1-pve: 4.2.6-26
lvm2: 2.02.116-pve2
corosync-pve: 2.3.5-2
libqb0: 0.17.2-1
pve-cluster: 4.0-29
qemu-server: 4.0-41
pve-firmware: 1.1-7
libpve-common-perl: 4.0-41
libpve-access-control: 4.0-10
libpve-storage-perl: 4.0-38
pve-libspice-server1: 0.12.5-2
vncterm: 1.2-1
pve-qemu-kvm: 2.4-17
pve-container: 1.0-32
pve-firewall: 2.0-14
pve-ha-manager: 1.0-14
ksm-control-daemon: 1.2-1
glusterfs-client: 3.5.2-2+deb8u1
lxc-pve: 1.1.5-5
lxcfs: 0.13-pve1
cgmanager: 0.39-pve1
criu: 1.6.0-1
zfsutils: 0.6.5-pve6~jessie

root@srv1:/# pvecm status
Cannot initialize CMAP service

root@srv1:/# systemctl restart corosync.service
Job for corosync.service failed. See 'systemctl status corosync.service' and 'journalctl -xn' for details.
root@srv1:/var/log# systemctl status corosync.service
● corosync.service - Corosync Cluster Engine
Loaded: loaded (/lib/systemd/system/corosync.service; enabled)
Active: failed (Result: timeout) since Thu 2016-04-21 08:22:50 CEST; 14s ago
Process: 34092 ExecStart=/usr/share/corosync/corosync start (code=killed, signal=TERM)

Apr 21 08:21:20 srv1 corosync[34097]: [MAIN ] Corosync Cluster Engine ('2.3.5.15-e2b6b'): started and ready to provide service.
Apr 21 08:21:20 srv1 corosync[34097]: [MAIN ] Corosync built-in features: augeas systemd pie relro bindnow
Apr 21 08:22:50 srv1 systemd[1]: corosync.service start operation timed out. Terminating.
Apr 21 08:22:50 srv1 corosync[34092]: Starting Corosync Cluster Engine (corosync):
Apr 21 08:22:50 srv1 systemd[1]: Failed to start Corosync Cluster Engine.
Apr 21 08:22:50 srv1 systemd[1]: Unit corosync.service entered failed state.

root@srv1:/var/log# ls -la /var/log/corosync/
drwxr-xr-x 2 root root 4096 Dec 4 13:55 .
drwxr-xr-x 11 root root 4096 Apr 21 06:25 ..
(empty)
root@srv1:/var/log# tail syslog
[...]
Apr 21 08:22:50 srv1 systemd[1]: corosync.service start operation timed out. Terminating.
Apr 21 08:22:50 srv1 corosync[34092]: Starting Corosync Cluster Engine (corosync):
Apr 21 08:22:50 srv1 systemd[1]: Failed to start Corosync Cluster Engine.
Apr 21 08:22:50 srv1 systemd[1]: Unit corosync.service entered failed state.
[...]

root@srv1:/# cat /etc/corosync/corosync.conf
logging {
debug: off
to_syslog: yes
}

nodelist {
node {
name: srv1
nodeid: 1
quorum_votes: 1
ring0_addr: srv1
}

node {
name: srv2
nodeid: 2
quorum_votes: 1
ring0_addr: srv2
}

}

quorum {
provider: corosync_votequorum
}

totem {
cluster_name: ZUOSCLUSTER
config_version: 2
ip_version: ipv4
secauth: on
version: 2
interface {
bindnetaddr: 192.168.13.51
ringnumber: 0
}

}

root@srv1:/# ssh root@srv2

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Thu Apr 21 08:05:16 2016 from 192.168.13.51
root@srv2:~#
(it's connection)

 

[t.lamprecht]

I had cluster with two nodes and it was damaged. Now I can't make backup and others.

How was it damaged? Did you tried to reinstall corosnyc-pve?

What happens if you start corosync in the foreground?

corosync
corosync -f # this starts it in the foreground.

 

[ookonek]

I am hot-headed, I worked on firewall... and it's happened!

I don't know how it is possible, but I didn't have sting with define ip and hostname another nodes in /etc/hosts.
I added it to each nodes, reinstall corosync -pve and now it is working.
What more I added third node for safety.

Thanks you very much t.lamprecht for your time & sorry!