[SOLVED] CEPH without switch

Oct 21, 2020
108
7
23
37
We are thinking of buying a high-density server with four nodes.

I would like to create a ceph cluster without having a switch.

The idea is to connect all the nodes to each other.
1701944922310.png
In the specific case with 25Gbits connections.

However, with a test system I can't get the system to work.
The problem is that I can't set the ports to be reached by the other nodes with a single IP.
According to my knowledge it should have been enough to set the Interface in Linux Bridge mode, but it does not work.
The only way I was able to make them communicate is with the Lunux Bound in broadcast setting, but I would like to avoid unnecessarily overloading the bandwidth by reducing it to a third.

Any suggestions?
 
Totally missed that page.
I used the "Routed Setup (Simple)" approach.


Note that multicast is not possible with this method
Can you explain to me what the downsides of this are?
 
hi, with simple you'll have no fallback...

ad multicast, I don't know, maybe it means you can not use some ceph features based on multipath. quick search reveals fe use_keepalived_multicast
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!