node cluster config

[dbrega]

I would like to create a cluster of three nodes with Ceph installed on both.
I would like to avoid using two switches because I don't need them, as the nodes are close to each other and are equipped with multiple 2.5GB network cards.
I was thinking of a configuration like the one shown in the image below.

I would like to know if anyone has any doubts about the correct functioning of this configuration.
Thank you in advance.

 

[UdoB]

I would like to know if anyone has any doubts about the correct functioning of this configuration.

Depending on your specific expectation... it will probably work. There is no magic involved

As soon as you need to tap in into one of the point-to-point networks you'll see the problem: you can not connect a fourth device, e.g. for trouble shooting.

2.5 GBit/s switches are cheap. Personally I would just buy one more. Instead of using the multiple NICs your way I would highly prefer to setup a separate connection for Corosync, a separate LAN for storage and a separate network for some (tagged only) VLANs. All of them using a switch, allowing for a fourth/fifth node to be attached next year...


Slightly off-topic in regard of your question: personally I would never run a three node Ceph installation in a productive environment. As soon as anything fails you are degraded - and you stay degraded as there is no redundancy to kick in and trigger self-healing - which is one of the killer-features Ceph offers.

I know several users do successfully use this construct, and of course: ymmv.

My own findings after a year of Ceph with 2.5 GBit/s and Mini-PCs: https://forum.proxmox.com/threads/fabu-can-i-use-ceph-in-a-_very_-small-cluster.159671/