Basic question on configuring ceph

ewuewu

Renowned Member
Sep 14, 2010
58
0
71
Hamburg
Hi Forum,

I'd like to install a ceph storage cluster. For this I have 3 machines. Each of them has an adaptec 8805 controller, four 4TB SAS Disks for Storage purposes and 2 small SSDs for OS.

My question is: Would it be better to create a raid 5 (from the four SAS disks) on each host and populate it afterwards as one single OSD or would it be better to create single disks an populate them as four OSDs on each host?

What would be best from the performance point of view?

Scenary one: 3 hosts with one prox 10TB (raid5) OSD - results in Ceph cluster with 3 OSDs
Scenary two: 3 hosts with four prox 4TB OSDs - results in Ceph cluster with 12 OSDs
 
Hi Forum,

I'd like to install a ceph storage cluster. For this I have 3 machines. Each of them has an adaptec 8805 controller, four 4TB SAS Disks for Storage purposes and 2 small SSDs for OS.

My question is: Would it be better to create a raid 5 (from the four SAS disks) on each host and populate it afterwards as one single OSD or would it be better to create single disks an populate them as four OSDs on each host?

What would be best from the performance point of view?

Scenary one: 3 hosts with one prox 10TB (raid5) OSD - results in Ceph cluster with 3 OSDs
Scenary two: 3 hosts with four prox 4TB OSDs - results in Ceph cluster with 12 OSDs
Hi,
inktank recommend no raid-controller - use path through to get 12 OSDs.
The IOs are limited by the OSD-Process.

But some people use such an config and the advantage is, that you only need an replication level of 2 instead of 3. Mean all data is stored twice and not three times!

See also here: https://www.mail-archive.com/ceph-users@lists.ceph.com/msg11661.html

Udo
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!