New setup : hardware advice

[Xendrei]

Hello,

After a successful testing of Proxmox/Ceph on a 3 PC Lab, I plan to install a production one on 4 Dell servers. I would like some advice on hardware I want to buy, and on compatibility (for example with the NIC Broadcom 57416 dual port 10GbE). Each server will have :

- 2 x Intel Xeon Silver 4116 2,1GHz 12C/24T
- 4 x 16Go RAM
- 240Go SSD SATA S4600 for Proxmox installation
- 2 x 2To HDD SATA 7200rpm for Ceph OSD
- Integrated NIC dual port 1GbE
- NIC Broadcom 57416 dual port 10GbE BaseT

I'll buy 1 switch 10GbE for Ceph network.

Is there something really wrong with this config ? And do you know if every components will work great with Proxmox/Ceph ?

Thanks for your help

 

[tschanness]

>2 x 2To HDD SATA 7200rpm for Ceph OSD

Your ceph will be queit slow. Try to add more (and faster) OSDs.

 

[udo]

Hi,
I'm not sure about the 10GB-Nics - I would go for Intel-Nics (Dell sold them too).

Depends on your workload, but 64GB Ram with this amount of cores sound much to less.
For my workloads I use 160 or 192 GB for such a server.

Udo

 

[czechsys]

2.1GHz = slow
10Gbe = worse than 10Gb sfp+
2x HDD = slow
16GB RAM = low

Result - bad ceph performance

 

[Xendrei]

Hello tschaness,
I was afraid of that, but price difference is really huge between enterprise SSD and HDD... Actually, I don't need high performance for disks (it will be a lot of light VM like DHCP servers, DNS, etc...). Does performance on HDD with Ceph will be really lower than without Ceph on the same disks ? Because today, those VM are on Virtualbox on HDDs 7200rpm and we don't have any performance trouble.

Udo,
Thanks for the advice on the NIC, I'll see with Dell if they can confirm compatibility with Debian for the Broadcom.
For the RAM, yes, it is undersize compare to Core amount. But we won't be able to upgrade the procs, and there is still plenty of free slots for RAM. So we plan to buy more RAM when needed in the future. That's the idea.

 

[Xendrei]

And for Ceph and OSD Bluestore, would I have better performances using a SSD for WAL and DB ? I have some Kingston SSD 120Go.

Thanks for your help.

 

[alexskysilk]

Hello,

After a successful testing of Proxmox/Ceph on a 3 PC Lab, I plan to install a production one on 4 Dell servers. I would like some advice on hardware I want to buy, and on compatibility (for example with the NIC Broadcom 57416 dual port 10GbE). Each server will have :

- 2 x Intel Xeon Silver 4116 2,1GHz 12C/24T
- 4 x 16Go RAM
- 240Go SSD SATA S4600 for Proxmox installation
- 2 x 2To HDD SATA 7200rpm for Ceph OSD
- Integrated NIC dual port 1GbE
- NIC Broadcom 57416 dual port 10GbE BaseT

I'll buy 1 switch 10GbE for Ceph network.

Is there something really wrong with this config ? And do you know if every components will work great with Proxmox/Ceph ?

Thanks for your help

there isnt anything wrong with it, although since you havent mentioned what you're planning to do with it no one can tell you if it will perform well. its like asking if a cup of water is enough without telling us if its for a mouse or an elephant.

As for ceph- no matter the application, 2 HDDs per node will NOT perform well.