Best switch for Ceph cluster network

[willybong]

Hi everyone,
I need a switch that It's be able to support hash Layer 3+4 for LACP for my cluster Ceph network.
Does anybody know to give me any suggests?

Many thanks!

 

[wolfgang]

Cumulus network switches can do that.

But for ceph, you need no Layer 3+4.
Layer 2+3 is quite good.

Ceph has different src and different dest so MAC and IP work well.

 

[spirit]

I'm pretty happy with my mellanox switches

 

[willybong]

Cumulus network switches can do that.

But for ceph, you need no Layer 3+4.
Layer 2+3 is quite good.

Ceph has different src and different dest so MAC and IP work well.

Thank you wolfgang, but MAC and IP address are always the same, or not? Is there a guide that describes it?
I noticed in this (https://forum.proxmox.com/threads/ceph-networking-optimization-on-gig.50192/) discussion that is better to use Layer 3+4.
Our customer had to use a switch with Layer 3+4 for Cluster Ceph Network (with 2 interfaces 40gps LACP) to obtain the best write/read performances.
What do you think about this?

 

[spirit]

Thank you wolfgang, but MAC and IP address are always the same, or not? Is there a guide that describes it?
I noticed in this (https://forum.proxmox.com/threads/ceph-networking-optimization-on-gig.50192/) discussion that is better to use Layer 3+4.
Our customer had to use a switch with Layer 3+4 for Cluster Ceph Network (with 2 interfaces 40gps LACP) to obtain the best write/read performances.
What do you think about this?

layer 3+4 should work better, as all qemu vms are doing multiple tcp connections to multiple osd. (so it'll balance on multiple links for differents osd on same machine).

But layer2+3 should works fine too if you have a lots of hosts.

 

[jlebherz]

Hey,

we are using juniper switches in VC for HA purpose, works very well with out 13 Node Ceph Cluster