Recommendation for Datacenter grade switch for running Ceph

B

bsinha

Member
May 5, 2022
83
0
11
What kind of storage switch should I look for running Ceph with a 6-node cluster?

Please suggest some models. At least the features that we need to look for will help too.

Thank you.
 
For new, Arista.

For less expensive option, Mikrotik.

Pick your speed, medium (wired and/or optic), and number of ports.

I use both without issues at 10GbE. They do make higher speed switches.
 
How much did you get on Write IOPS? We have used Ceph SSD with 4 node servers. Each of them is equipped with enterprise drives (Kingston DC500M - Claiming to provide 75000 random write IOPS). However, when we check it from Ceph, we are barely getting 3K write iops.

And we assume it is for the switch which uses Store-and-forward technology. (This is what by default a network-grade switch does),

Hence, we are requesting to suggest a data center grade switch so that we can at least get around 20K write IOPS.
 
Not sure if the switch will help you on the IOPS side, but if you want new.. Arista. For used I can highly recommend Mellanox. You can get SN2410's for around $ 1500 on ebay (if you're lucky) giving you 48x25 gbit plus 8x100 gbit, very low latency.

We've been running these in production for years now, no complaints whatsoever.
 
dignus said:
Not sure if the switch will help you on the IOPS side, but if you want new.. Arista. For used I can highly recommend Mellanox. You can get SN2410's for around $ 1500 on ebay (if you're lucky) giving you 48x25 gbit plus 8x100 gbit, very low latency.

We've been running these in production for years now, no complaints whatsoever.
Click to expand...
The switch will help in better performance. The SN2410 has the smart cut-through feature. Thank you for your suggestion. It helped me.
 
Yeah, but not stacked as in Cisco or FS style. You configure both switches with an MLAG In between them. After that you create a port channel for each port on each switch. We have this running in production.
If you go the used route, keep in mind you want to replace the disk.
 
We shall run 7 node Proxmox Ceph clusters. 48 port SN2410 would be overkill. Can we for SN2010?

We are planning to take 2 x SN2010 and put them in stack. Will it be good?
 
Sure. But on the used market you will find that they might be more expensive than it's bigger brother :)
If I had the choice I'd go for SN2700's, very attractive price point. Overkill? Yeah, for now, you never know in the future. You can get break out cables to 4 x 25.
 
You must log in or register to reply here.
Top Bottom