Ceph in proxmox cluster

[bibax]

Hi,

I need to have some advice on a new architecture.
I want to install a cluster (with 3 nodes) but I want Ceph on only 2 of them.
Is it possible to have a shared storage only on 2 nodes (with HA configured to restart vm on the second with Ceph) and an other node with only local storage?

Thank you in advance for your advice!

Regards

 

[udo]

Hi,

I need to have some advice on a new architecture.
I want to install a cluster (with 3 nodes) but I want Ceph on only 2 of them.
Is it possible to have a shared storage only on 2 nodes (with HA configured to restart vm on the second with Ceph) and an other node with only local storage?

Thank you in advance for your advice!

Regards

Hi,
yes and no.

You can do that, but ceph with two nodes only isn't HA.

Udo

 

[bibax]

Hi,

Can you explain why?
If i have 3 nodes, the quorum will be good to know if az node is down.
And if server are on the node which is down, all server (virtial) will go on the other server which have ceph (if i specify on which node migrate virtual server)?
Is there a limit directly on ceph?

 

[bibax]

Hi,

Do you think that this architecture will work :
- A proxmox cluster with 3 nodes (P1, P2, P3)
- Setting up a mon on each node
- Add only osd on 2 nodes (P1 and P2) so the last node (P3) has no osd
- Set a rule to say that all VM will restart on the node wich have the other osd : vm on P1 will restart on P2 and vm on P2 will restart on P1

With that, we have the quorum of the cluster and 3 mon to monitor the ceph.
Do you think that if P1 will be down, all vm will be ok on P2 and the ceph will be ok?

I will have a journal disk on P1 and P2 and 3 SSD Disk with 512Go on each.

Thank you in advance for your advice.

Regards

 

[Kamyk]

Hi,

I'm looking for answer too. I think I have the same problem. I have two the same servers with OSD on it and mac mini for quorum. When I setup HA on VM I can not do anything with it

bibax - how you setuo your VM's? Does HA work for you?

Best,
Rafal

 

[fabian]

ceph with only two OSD nodes is a bad idea, both performance and reliability wise. three should be considered the minimum - more is better though.

 

[Kamyk]

I agree. But I have two normal servers and one mac mini there for quorum. I can not add more now. I will in future but not now

Rafal

 

[tschanness]

It would work when you edit the placement in the ceph config. This was discussed before.

Regards,
Jonas

 

[Kamyk]

Could you be so kind and point me to that? I wish to have that two servers like a masters and mac mini not.

Best,
Rafal

 

[tschanness]

I'm too lazy to search the Forum right now but you will find that to do here:
https://websetnet.com/how-to-configure-single-node-ceph-cluster-to-run-properly/

Regards,
Jonas

 

[Kamyk]

I will check it. Thank you for that.

Best,
Rafal