4 Node Proxmox Cluster and CEPH

[jmlsantos75]

Dear all,

Greetings to this fantastic community.

I have a 3-node Ceph cluster with four NVMe OSD on each node and a dedicated 40gbit/s network for Ceph.

PVE1: 4 osd + mon + mgr (active)
PVE2: 4 osd + mon + mgr (stand by)
PVE3: 4 osd + mon + mgr (stand by)

I want to add the 4th node and install Ceph, too.

I read that the number of Ceph monitors should be odd.

I am considering installing Ceph on a 4th node with 4 OSDs, but without a monitor or manager. Is this a viable configuration?

Best regards,
Jorge

 

[mgabriel]

Yes, you can do that and I'd recommend keeping 3 MONs and MGRs.

Please keep in mind that if you lose one server, Ceph will immediately recover the missing objects on one of the remaining nodes. that leads to a higher storage usage and can lead to full OSDs.

It depends on your goal:

- if you want to get more usable disk space, add a fifth server or add some more storage to the first three.
- if you want to get self-healing in case one of your servers fail, 4 servers does this.

 

[jmlsantos75]

Dear mgabriel,

Thank you. The main goal is to get Self-healing! I'm going to add the 4th node like this
PVE4: 4 osd

Thanks
Jorge