Advice for new proxmox-ceph cluster

[David Calvache Casas]

Hello All.

I'm going to buy new hardware to build a new proxmox cluster, and I would like you to validate the configuration:

The idea is to put 5 machines in cluster, each with 4 network ports at 10Gbps. I will also buy 2 switches of 48p 10Gbps in SFP + connected in LACP to have tolerance to failures and load balancing.

The servers have 2 SSD (raidz in mirror) to boot the S.O and for the use of the monitors (3). As storage unit (only one at the moment) a nmve disk of 3.2 TB

These are the characteristics of the servers:

5X SuperServer 6029U-TRTP
* Intel Gold Processor SP 5118, 12 core / 2.30Ghz
* 96 GB Ram Memory 2666Mhz (16GBX6)
* 2 SSD 2.5 hot removable located on the back of the machine for S.O. - Samsung SM863 480 GB
* 1 NMVE 2.5 "SAMSUNG PM1725a 3.2 TB
* 12 Bahias 3.5 ", of which 4 support NMVE
* LSI 3008 controller HBA mode with 8 SAS / SATA disk capacity
* 1000W redundant power supply range Titanium hotswap
* 4 SPF + 10GB network cards

For the switches I thought of two HUAWEY CE6810-48S4Q-LI a 48p 10Gb with uplinks to 40Gb.

The idea is to run the virtual machines, the ceph and the monitors on those same servers. These servers will run the critical infrastructure of the business.

Do you think it is an adequate configuration? Would the NMVE unit divide it into several OSDs?

It would be ideal to hire proxmox support for the cluster if they support this hardware, can someone from the staff tell me, if it is possible?

I would greatly appreciate your input.

 

[David Calvache Casas]

nobody?

 

[tom]

I do not see what you plan for OSDs, but the hardware looks not that bad. But without knowing all details its quit hard to tell which part can be improved.

And I would add (much) more RAM, but this depends on your needs and the settings.

 

[David Calvache Casas]

Hi Tom!

Not spinner OSDs involved. At least at the early production time.
Only the Samsung pm1725 3,2 TB nmve will be used as OSD.. ( i think i must partitionate it in three OSD to enhance perfomance, but i dont know if this, is a waste of resources and time)

If the ceph cluster goes well, i adquire more nmves to scale the usable size.

The main goal of this cluster is to run critical mvs (linux and windows), i do not expect use lxc here.

I bought 96 Gb of RAM per machine, because the example cluster published in video has only 64 GB?. To rise the mem i need to go to 192GB, because that processors have 8 memory channels.

Do you expect good perfomance from this setup?

Thanks

 

[tom]

Hi Tom!

Not spinner OSDs involved. At least at the early production time.
Only the Samsung pm1725 3,2 TB nmve will be used as OSD.. ( i think i must partitionate it in three OSD to enhance perfomance, but i dont know if this, is a waste of resources and time)
...

nvme for OSDs, fast!

we did some performance tests with 3 nodes - each with a Intel P3700 800 GB - with very good results. I cannot comment on your network - we used Mellanox 100Gbit for these tests.