3 site Ceph cluster

[chitambira]

I am not sure if this is the right forum to post. I am looking for suggestions on the best way to configure Ceph for a hyper-converged infrastructure using 5 nodes in 3 datacenters, each node with 2 SSDs. (2 DCs will have 2 nodes, and a 3rd DC will have 1 node - I can increase its OSD capacity if need be)

Many thanks

 

[gurubert]

How far away are the datacenters? Ceph is very affected by latency.

 

[chitambira]

How far away are the datacenters? Ceph is very affected by latency.

They are within 2 miles of each other and latency should be within 2ms. I am aware that I can enable stretch cluster if one site latency gets too big, but for now lets assume its like 3 separate machine rooms within the same LAN. Many thanks for responding.

 

[czechsys]

Check https://pve.proxmox.com/wiki/Cluster_Manager#_cluster_network and other network related things https://pve.proxmox.com/wiki/Deploy_Hyper-Converged_Ceph_Cluster