How-to resync cluster after network outage ?

[Florent]

Hi everyone,

Yesterday I had a big network issue, shutting down all communications between my nodes during a few jours.

All VMs are still up on every node, but the whole cluster is desynced.

In the GUI, all nodes appear in red.

I did that on one node :

pvecm expected 1
service cman restart
service pve-cluster restart

So now when I go on the GUI on it, it appears green.

But impossible to resync all other nodes to the cluster.

Configuration has not changed when network was out, so I think they are all up-to-date. But I can't administrate them and view VMs.

Here is pveversion :

proxmox-ve-2.6.32: 3.1-109 (running kernel: 2.6.32-23-pve)
pve-manager: 3.1-3 (running version: 3.1-3/dc0e9b0e)
pve-kernel-2.6.32-22-pve: 2.6.32-107
pve-kernel-2.6.32-23-pve: 2.6.32-109
lvm2: 2.02.98-pve4
clvm: 2.02.98-pve4
corosync-pve: 1.4.5-1
openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.0-1
pve-cluster: 3.0-7
qemu-server: 3.1-1
pve-firmware: 1.0-23
libpve-common-perl: 3.0-6
libpve-access-control: 3.0-6
libpve-storage-perl: 3.0-10
pve-libspice-server1: 0.12.4-1
vncterm: 1.1-4
vzctl: 4.0-1pve3
vzprocps: 2.0.11-2
vzquota: 3.1-2
pve-qemu-kvm: 1.4-17
ksm-control-daemon: 1.1-1
glusterfs-client: 3.4.0-2

What can I do to resync that all ? Of course, I don't want to shut down VM or nodes (I can't move VM to other node !!).

Tell me if you need more information on my config.

Thank you a lot.

Flo

 

[Florent]

I'm sorry for the post, I have found the solution.

Just run on every node :

service cman stop; service pve-cluster stop

Then

service pve-cluster start; service cman start