[SOLVED] Recommendation small Ceph setup

[BJ78945]

Hello,

at the moment I use GlusterFS in a low setup (EVO SSD and only 2GBE between the nodes). I want to increase performance and want to build a 3 node Ceph cluster for testing. I have the following devices:

6x SM883 960GB SSD
10x EVO 870 250GB SSD
12x different 300GB SAS 10k HDD

Alle three nodes will be connected with 40GBE DAC in a mesh setup.

I want to get a good performance. So I want to use two EVO SSD (RAID1) for the OS and the SM883 for OSD. I think I don't use the SAS drives. I read that 12 OSDs will be recommended. Also see in the ceph docs, that for faster drives more than one OSD could bring better performance. Is it a good idea to setup 2 OSD per SM883 drive, so I have 12 OSDs in the three node setup? Or perform it better to use the 12 SAS and for 2 SAS one SSD for journal? Or could it also be better to use the cheap EVO SSD as OSD and the SM883 as journal?

If all of them are not really a good choice, which drives you could recommend? 1,5TB useable storage would be enough for the beginning. Per node I have space for 8 2,5" devices.

Best regards,

Alex

 

[Felix.]

Better use 2 of the SAS HDDs for OS instead of the consumer SSDs as those would wearout pretty quick.

I'd use 2 ceph pools, one using the SM883's as those are proper datacenter drives and another pool using the EVOs.
Just be aware that the EVOs may not live very long, depending on your write load, also the SM883's will probably have a way more stable performance.

 

[BJ78945]

Thank you very much for your answer. One question left. Is it a good idea to put 2 OSD on one SM883? Or should I buy more SM883 to become 4 OSD per node? I saw that at least 12 OSD should be used.

 

[czechsys]

If you aren't skilled with ceph, the better way will be 1 osd per ssd.

 

[BJ78945]

Ok thank you