CEPH client only node

[kissze]

Hello,

I have a hyperconverged Proxmox+CEPH cluster witch is working well. I have 2 networks currently (i know it would be better to separate ceph public from vm traffic, but currently i only have 4x10g per node), 1 for ceph cluster on a dedicated bond and 1 for ceph public and VM traffic on a dedicated bond. Now i have a fourth server and i want to join to Proxmox cluster, but not to the CEPH cluster, because it no makes sense with the 2/3 replicas configured. Can i join the new node without adding the ceph cluster network to it to work as a client only with another three nodes?
Or i have to configure the cluster network also to work everything as on the other nodes?

Cheers,
Zoltan

 

[alexskysilk]

Can i join the new node without adding the ceph cluster network to it to work as a client only with another three nodes?

Joining a PVE cluster does not mean you're joining the ceph cluster. IF you had the right connectivity it would just work, but its fine if you dont.

Just make sure to NOT include that node in workloads that involve the ceph store.

 

[kissze]

Joining a PVE cluster does not mean you're joining the ceph cluster. IF you had the right connectivity it would just work, but its fine if you dont.

Just make sure to NOT include that node in workloads that involve the ceph store.

Yes, i know that these things are different. But with default install there is no CEPH installed i think. So i can join the PVE cluster, but what will happen with ceph storage? I have to install manually the packages for CEPH or install from the GUI? As far as i remember if im installing from the gui, then some configuration will be created by PVE and the question is that how can i tell the GUI to use only the public network, no cluster network? Maybe its confusing, but i hope its understandable

 

[alexskysilk]

But with default install there is no CEPH installed i think.

from the ceph repo. ceph client packages are provided with debian.

So i can join the PVE cluster, but what will happen with ceph storage?

Nothing. Why would you expect it to?

how can i tell the GUI to use only the public network,

just enable and install the proxmox ceph repo (pveceph install) so it can use the ceph configuration provided in /etc/pve (easiest way.) it would just work.

 

[kissze]

from the ceph repo. ceph client packages are provided with debian.


Nothing. Why would you expect it to?


just enable and install the proxmox ceph repo (pveceph install) so it can use the ceph configuration provided in /etc/pve (easiest way.) it would just work.

Thanks, i will try this method (pveceph install)

I mean what will happen with the storage defined in the pve cluster. Will show up as unreachable for example.

I know that there will nothing happens with the ceph storage/pool itself

 

[UdoB]

i want to join to Proxmox cluster, but not to the CEPH cluster, because it no makes sense with the 2/3 replicas configured.

Why doesn't it make sense???

Four nodes are better than three ;-)

Beside the capacity aspect think: with three Ceph-Nodes and one failing there is NO CHANCE for self-healing as the "failure domain" is "node".

But with four Ceph-Nodes, still with size=3/min_size=2, self-healing will be possible! When one node fails the required three are still available.

(( Of course only one node may fail. This was true for three nodes and still is true for four nodes. ))

Some more aspects in this regard: https://forum.proxmox.com/threads/fabu-can-i-use-ceph-in-a-_very_-small-cluster.159671/


Also note that PVE-quorum and Ceph MON majority is completely independent. For PVE you may add a separate "QDev" to get five votes, for Ceph such a thing does not exist.