How to assemble cluster after a network failure

[aasami]

Hello all,
may be a noobish question, but I have a fire on the roof (metaphorically).

We had a network issue, and all nodes stopped seeing each other at the same time.
After we resolved the network issue, all cluster nodes seem to be on their own:

# pvecm status
Quorum information
------------------
Date: Wed Dec 5 20:36:21 2018
Quorum provider: corosync_votequorum
Nodes: 1
Node ID: 0x00000003
Ring ID: 3/2760
Quorate: No

Votequorum information
----------------------
Expected votes: 4
Highest expected: 4
Total votes: 1
Quorum: 3 Activity blocked
Flags:

Membership information
----------------------
Nodeid Votes Name
0x00000003 1 10.20.30.35 (local)
#

So the question is simple.
How can I rebuild the cluster?

Thank you very much for any quick answer.

 

[wolfgang]

Hi,

Please check that multicast ist still working.
you can use omping for this.
https://pve.proxmox.com/wiki/Multicast_notes

 

[Oyabi]

Hi,

I have the same problem.
Multicast is ok on all my nodes but there every host is on a separate mini cluster.
Can you help me please?

Regards.

 

[Stoiko Ivanov]

* try restarting corosync
* check and read the logs - `journalctl -r`, `journalctl -f`
* try to check the latency and connectivity with omping
https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_cluster_network

 

[Oyabi]

Ok guys it was a firewall issue.
I flush my iptables rules and everything is ok.