Three node Ceph cluster: physical switching vs. full mesh network

[sherminator]

Hi,

I just learned from here that it's possible to connect the nodes of a Ceph cluster directly to each other.
I wonder if that would increase the I/O performance of the cluster, because there are no physical switches in between.

For our upcoming project both (physical switching and a full mesh network) would be possible. What are the advantages and disadvantages of these two ways?

Thanks a lot for sharing your experience and many greets
Stephan

 

[alexskysilk]

The advantage for mesh networking is pretty obvious- no need for switching gear. the disadvantage- it does not scale; you cant add additional nodes beyond the three. There is also a minor operating cost (performance wise) for having the switching performed on the nodes but its not a showstopper. Practically speaking, mesh networking is only useful for lab purposes.

 

[sherminator]

Ok, so letting the switches do the switching is more performant than giving this job to the hosts - makes sense!
So in your opinion I should always do classical switching in a productive environment?
Are there other opinions about this?

Thanks a lot and many greets
Stephan