Asymetric cluster with Ceph; best practise for quorum

T

Tom Schmidt

New Member
May 8, 2024
11
1
3
Hi all,

I'm just interested in your opinion.

We currently have a "streched" Proxmox-Cluster with 3 nodes using Ceph. Each server holds 7 OSD. Two servers a located in our main datacenter (with emergency power generator etc.) one server runs in our backup datacenter (only ups buffered for about 3 hours). Both datacenters a connected via a 100GB switch-cluster.

We have to expand our compute and memory ressources by adding 2 nodes whithout osd (1 server in the main datacenter, one in the backup datacenter. Buying 14 more NVMe is out of the question right now.)

So we need to make sure that at all time at least 2 servers with osd a running, otherwise there should be no quorum.

My idea is to give the servers that host osd 2 votes each, while the servers without osd only get on vote. So expected votes would be 8; quorum would reached at 5 votes. So whenever 2 ceph nodes fail there can't be a quorum.

Am I missing something?

Thomas
 
ness1602 said:
Why are you putting those two nodes in the same cluster as those before? Why not just for starters, create them in separate clusters and work with that?
Click to expand...
First thing that comes to mind is live migration. Also these two nodes have to use the ceph storage, and maybe later will get osd
 
If 2 Ceph nodes fail it’ll be below 2 chunk copies and out of Ceph quorum so the cluster quorum is a bit irrelevant. Storage will be read only at best. Failing cluster quorum at that point would trigger a reboot and I’m not sure that helps…?

You may be able to make the new nodes Ceph monitors only, I’m not sure. But you’d still have only one copy of the data.
 
You must log in or register to reply here.
Top Bottom
Back