Planning Ceph 3 Nodes - 6 OSD vs 3 Hardware Raid

[Mihai]

Hello all,

I don't have any experience with Ceph and I wanted to get some people's opinions on this:

• 3 Node Cluster (2x R510, 1xR520)
• Storage to be used for light VM usage.

Here are the two options I'm considering:

OPTION 1

6 spinning hard disks for OSDs, 2 per node. (6 total OSDs)
3 SSD journal disks, 1 per node.

Drawbacks:
If a disk fails, I have to use the re-build using ceph gui/commands.​

Advantages:
?​

OPTION 2

6 spinning disks but in 3 RAID1 mirrors using Dell's raid cards, 1 raid per node. (3 total OSDs)
3 SSD journal disks, 1 per node.

Drawbacks:
?​

Advantages:
Trivial to replace a failed disk without having to mess with ceph.​

 

[PigLover]

Separate OSDs. Don;t bother with the Raid1. Ceph will be doing your replication, etc, and the raid layer will just reduce your overall capacity (raid1 local replications cuts capacity in half, but Ceph will still do replication across the hosts) with limited performance gains. Ceph works more effectively with more OSDs exposed to it (even as proposed 6 OSDs is a pretty small Ceph deployment).

 

[Mihai]

Oh okay, thanks for that info.

Also, I want to make sure that the data is distributed over 3 OSDs on the three separate hosts.

Is that done automatically if I specify the replication count to 3?

Could it be possible that CEPH may use 2 OSDs on the same host as part of the 3-replication count?

 

[czechsys]

Oh okay, thanks for that info.

Also, I want to make sure that the data is distributed over 3 OSDs on the three separate hosts.

Is that done automatically if I specify the replication count to 3?

Could it be possible that CEPH may use 2 OSDs on the same host as part of the 3-replication count?

Ceph replicate per host by default, not per OSD.

 

[Mihai]

Ceph replicate per host by default, not per OSD.

Excellent! Thanks.