Selfmade Server

[Afox]

Hello,

can anyone recommend some manufacturers for building a professional selfmade server? It would be nice if some concrete products would be named for following pieces:

• Case (2U Rackmount, 24x2,5", redundant Power supply)
• Mainboard (Intel Xeon Processor, min. 2 Sockets, IPMI, min. 2 NIC)
• CPU (Intel Xeon, good value-for-money ratio)
• RAM (DDR4)
  
• RAID-Controller or HBA (has to be able to deal with 24x SSD)
  
• SSDs (min. 240GB, good value-for-money ratio)
• SSDs for Ceph
  
• 10Gbit fibre channel (to handle Ceph or any other storage solution)

I am searching for a cheap, reliable solution.

Thanks and regards,

Afox

 

[spirit]

for ceph, if you need to do a lot of iops or have low latency, I recommand you fast frequency intel cpu (3ghz).

for ceph ssd, use datacenter ssd. (i'm using intel s3520 && s3610 1,6tb, they works fine).
if you really need 24 ssd in one box, use mutiple controllers (1controller for 8disk, direct attach, no expander backplane).

maybe look at supermicro for case, motherboard.

 

[mir]

if you really need 24 ssd in one box, use mutiple controllers (1controller for 8disk, direct attach, no expander backplane).

This is not needed anymore: http://www.avagotech.com/products/storage/host-bus-adapters/sas-9305-24i

 

[Afox]

I like the linked Controller!

Also the other hints are very useful. What about the 2U case? can you recommend any manufacturer?

Talking about high frequency CPU: can you recommend an up-to-date and affordable Xeon for this purpose? Besides Ceph I also want to run the VM machines on this servers so it should be able to handle both in a performant manner.

 

[alexskysilk]

If I were you, I'd start the other way.

define your operations goals. How many VMs, how much resources they require, what are my performance requirements, what are my fault tolerance threshholds?

then, as for budget: how much revenue is being generated on my configuration? what are the consequences of the service being down or worse, data lost? your budget should reflect the ratio of revenue lost : revenue generated

Why are you specing out ceph? do you have existing nodes to include in your cluster, or were you planning to deploy multiple nodes?

Lastly- how much headache are you prepared to handle? if you build the box yourself any failure will be your fault and your responsibility. Do you really want that? you didnt spec any BMC capabilities; are you planning to house this server in your home or will you run to your datacenter any time there is a problem?

In my experience, custom building whiteboxes for production use not only doesnt save any money but adds headache too.

 

[mir]

These new DC SSD from Intel looks interesting: http://www.newegg.com/Product/Produ...BC9182&cm_re=intel_ssd-_-20-167-389-_-Product

 

[Rhinox]

What about the 2U case? can you recommend any manufacturer?

For self-built servers I've been using SuperMicro cases/mobos for some time. They are quite solid. The only thing I hate are some bloody small parts I have to order separately (fan-holders, riser-cards, hdd/ssd mounting-kits, etc). It happens to me quite frequently I forget something...

Recently I ordered Xeon-D SuperMicro motherboard together with recommended case (checked their website). Just after it arrived I found I could not mount mobo in case, because the standard included backplane was not compatible with this mobo! So I had to order it extra, payed $3 for backplane, $10 for delivery, and lost one week...

I also tested some low-cost cases but I can not recommend it. Manufacturing quality is generally lower, design not very clever, etc.

 

[Afox]

ok, thanks. Supermicro website is kind of a mess but I will find my way through it, I guess.

Another thing that I didn´t list is a fibre channel or infiniband switch for ceph. For start I could also imagine a Mesh network like described here: https://pve.proxmox.com/wiki/Full_Mesh_Network_for_Ceph_Server
Nevertheless I´d like to ask if someone can recommend any affordable switch for this purpose. It should have min. 10Gbit and low latency. I would also choose infiniband over fibre due to the lower latency.
What about the other Network-connections? Is copper "enough" for this? What bandwidth should it have?
For start I am thinking of a 3 node HA Cluster with Ceph.

 

[Afox]

ok, I change my question: how do you connect your Ceph nodes? Can you give some advice for low latency hardware?

Regards, Afox