Ceph Crush Rule for HDD

[sahostking]

Hi

So we have been using the crush rule "replicated rule" for ssds only.

I want to now add hdds to each server so we have some "slow" storage with tons of diskspace.

Are the steps as per below:

1. Run the following on any node: ceph osd crush rule create-replicated replicated_hdd default host hdd
2. Create a new pool and assign the above rule to that pool
3. Add hdd OSDs which would be auto added to that pool

Is it really that easy?

Thanks

 

[sterzy]

Hi,

Is it really that easy?

More or less, yes, but step 3 should probably be step 1 and make sure that all HDDs get added with the class "HDD" otherwise your rule might not work as intended.

I'd also set either a target ratio or size when creating the pool so that the autoscaler can figure out how many PGs to allocate. If all you want is one pool on all the HDD OSDs then you can just set a ratio of 1.

 

[aaron]

Before you add the HDDs, create a rule targeting the SSDs and assign it to your current pool. Otherwise, the current pool will be rebalanced over both, HDDs and SSDs as the default replicated_rule will use any OSD, no matter the device class.

And once you assign device specific rules, all pools must be assigned to either one or the other device class. The "replicated_rule" should not be used anymore as that would create an overlap between device classes which will hinder the auto scaler to work properly.

 

[vincentjanv]

Thanks Aaron, this also helped me. I do however face an issue on 6.4.15 when trying to add an HDD rule, really strange. A new SSD rule worked perfectly:

ceph osd crush rule create-replicated replicated_hdd default host hdd



Error EINVAL: device class hdd does not exist

 

[aaron]

Creating the HDD rule needs to have at least one OSD of that device class already present.