Cluster stopped to work

[f4242]

Hello,

I have a cluster of 5 nodes. Yesterday, I upgraded all nodes to corosync 3. Then I upgraded 2 of the nodes to Proxmox 6.0. So I now have 3 nodes at 5.4 and 2 at 6.0. Everything was working great yesterday after this upgrade.

Today I come back to work and I see that all my nodes are isolated and don't see others nodes. When I do "pvecm status" on one of the nodes, the cluster doesn't quotate (only see itself), but on the others nodes the cluster have quorate (but the nodes are not available on the web UI). The command is slow to return in both cases.

What could be wrong?

Thanks.

 

[f4242]

Cluster resumed once I restarted corosync on all nodes. Will see if the problem occurs again...

I tried to see corosync logs, but the directory /var/log/corosync is empty (only have a file named ".empty").