Basic question on configuring ceph

[ewuewu]

Hi Forum,

I'd like to install a ceph storage cluster. For this I have 3 machines. Each of them has an adaptec 8805 controller, four 4TB SAS Disks for Storage purposes and 2 small SSDs for OS.

My question is: Would it be better to create a raid 5 (from the four SAS disks) on each host and populate it afterwards as one single OSD or would it be better to create single disks an populate them as four OSDs on each host?

What would be best from the performance point of view?

Scenary one: 3 hosts with one prox 10TB (raid5) OSD - results in Ceph cluster with 3 OSDs
Scenary two: 3 hosts with four prox 4TB OSDs - results in Ceph cluster with 12 OSDs

 

[dietmar]

IMHO you need at least 12 OSDs (ceph.com recommends 100+).

 

[udo]

Hi Forum,

I'd like to install a ceph storage cluster. For this I have 3 machines. Each of them has an adaptec 8805 controller, four 4TB SAS Disks for Storage purposes and 2 small SSDs for OS.

My question is: Would it be better to create a raid 5 (from the four SAS disks) on each host and populate it afterwards as one single OSD or would it be better to create single disks an populate them as four OSDs on each host?

What would be best from the performance point of view?

Scenary one: 3 hosts with one prox 10TB (raid5) OSD - results in Ceph cluster with 3 OSDs
Scenary two: 3 hosts with four prox 4TB OSDs - results in Ceph cluster with 12 OSDs

Hi,
inktank recommend no raid-controller - use path through to get 12 OSDs.
The IOs are limited by the OSD-Process.

But some people use such an config and the advantage is, that you only need an replication level of 2 instead of 3. Mean all data is stored twice and not three times!

See also here: https://www.mail-archive.com/ceph-users@lists.ceph.com/msg11661.html

Udo