CEPH network configuration (3 servers without switch)

[lexz]

Good afternoon,

Hope someone over here can give me some advice.
We are testing with a 3 node Proxmox cluster with Ceph.

All three machines have a dedicated NIC for Ceph with 2x 10Gbit.
Our idea is to connect all server directly to each other, without a switch.

Server 1 eth0.1 to Server 2 eth0.1
Server 1 eth0.2 to Server 3 eth0.1
Server 2 eth0.2 to server 3 eth0.2

Do we need to use a bond? Or what is the best way to configure this kind of setup?

Thank you in advance!

 

[lexz]

I think I've found the answer in the documentation.
https://pve.proxmox.com/wiki/Full_Mesh_Network_for_Ceph_Server

But if there are other tips, let me know!

 

[kkjensen]

How did this go? I have the same sort of setup but with 4 machines and am hoping I can bridge the interfaces to form a 10Gbe ring. I know it would be better with a switch in this case but I don't have enought 10Gbe SFP ports to available at the moment.