Ceph with two nodes....

tom

Proxmox Staff Member
Staff member
Aug 29, 2006
13,647
420
83
its possible but it makes no sense. minimum is 3 nodes.
 

dietmar

Proxmox Staff Member
Staff member
Apr 28, 2005
16,484
315
83
Austria
www.proxmox.com
Thank to answer Tom... Can you point me some docs?? I try it through web interface, but no way! I try create a pool with size/min like 2/1 but after poweroff node 2, ceph stop to work...
quorum...

If you run 2 ceph monitors, you will loose quorum if one dies.
 

tom

Proxmox Staff Member
Staff member
Aug 29, 2006
13,647
420
83
perhaps I do not myself clear
can I use the cluster quorum configuration to provide quorum to a ceph server, or cluster quorum and ceph quorum are different things....
No. I suggest you start reading here - http://pve.proxmox.com/wiki/Ceph_Server#Further_readings_about_Ceph

It makes no sense to help to setup a useless setup and to point others to this. If you want Ceph, use at least 3 nodes.

It also possible to experiment with virtual Proxmox VE Ceph servers - especially if you do not have the 3 nodes for testing.
 

gosha

Member
Oct 20, 2014
274
15
18
Russia
Yes, of course. If quorum present it will work.
If You don,t need HA, even two nodes will work. But in this case if one node from two will down, the other node don't will work. Quorum - more then half nodes. This will be - 2 nodes for 2-node cluster (and 2 nodes too for 3-node cluster, for example). 3-node cluster may be HA, but 2-node cluster - don't.

---
Best regards!
Gosha
 
Last edited:
  • Like
Reactions: fibo_fr
Apr 20, 2015
89
1
8
Hi,
three mon & 2 osd-server will work and give you a bit HA.
You need to set size "2" to your pool.
So you have a second copy from your data.
Its not perfect and not recommended, but works good.
You have to look carefully, that your osds have enough space left for taking copys from other osds if some fail.
I have used this a time. I could restart a osd-server without stopping vms on the other nodes..
But now we have three node-setup...
Markus
 
  • Like
Reactions: fibo_fr

fibo_fr

Member
Apr 21, 2014
40
2
8
Remoulins, France
Thx.
Got it working after some fights, trials and errors due to my network config... which pveceph DOES NOT LIKE if you have anything special.

RTFM is correct... but slightly obscure. Secret weapon is to manually edit ceph.conf and give explicitly all the values needed in public network / cluster network in [global] (2 sets of 3 addresses/32) and in the [osd.x] (2 sets of 2 addresses)
 

Kei

Member
May 29, 2016
86
2
8
32
three mon & 2 osd-server will work and give you a bit HA.
You need to set size "2" to your pool.
Hi Mark, could you please provide us with additional info about why having only two OSD nodes would not give you full HA? If I understand correctly...
 
Apr 20, 2015
89
1
8
Hi,
if one node goes down all pg - copies will be placed on the remaining node.
This can result in a very high load for a long time & evtl. (too) full disks.
If then an disk dies your data could be lost / hard to repair.
(For maintenance you can set your osds "noout")

Markus
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!