Proxmox Cluster with 2 nodes on OVH

[muko]

All,

I'm stuck on the installation of two nodes (one in Canada and one in France) in cluster.
The setup went fine, the upgrade too. On both nodes.

Now the steps I'm following are:

1- To properly configure the eth interfaces attached to the vRack
2- To create the cluster
3- To join the second node into the cluster
4- Install and setup Ceph

Cluster and Ceph are using the interface connected on the vRack, hosts files are properly configured and I've double checked that hostnames can be resolved.
MTU are exactly the same (I tried both 1500 and 9000)

Apparently everything is configured fine but the cluster loses quorum almost every 10 seconds and I still get Ceph Ealth issues.

Any suggestion on how to solve the issue?

EDIT: I also tried to add an external quorum device

 

[Klaus Steinberger]

A Cluster with only two nodes will always provide you with trouble -> the thing is called split brain

A long distance could also give trouble (latencies in corosync are bad)

And now:

CEPH needs at least 3 nodes!
CEPH is very latency dependant. If you want to see a decent performance for your VM's create a 3 node cluster at both sites.

you can use remote mirroring in CEPH, but keep latencies in mind, as long as nobody invents hyper light speed transmissions ;-)

 

[muko]

A Cluster with only two nodes will always provide you with trouble -> the thing is called split brain

A long distance could also give trouble (latencies in corosync are bad)

And now:

CEPH needs at least 3 nodes!
CEPH is very latency dependant. If you want to see a decent performance for your VM's create a 3 node cluster at both sites.

you can use remote mirroring in CEPH, but keep latencies in mind, as long as nobody invents hyper light speed transmissions ;-)

Thanks but now I only have two nodes. Are there any issue in having the cluster without ceph and install it later when having the third node?
Maybe I will not use migration but this could not be an issue

 

[Klaus Steinberger]

If you do not need migration between the two continents use them as two separete nodes. Clustering over long distance is a very picky thing