ceph device classes hdd/nvme

[jsterr]

On which drives does the default ceph crush rule "replicated_rule" place their data on? so if I create replicated_rule_nvme, I know data gets on the nvmes. But how do I know, on which devices the replicated_rule (default creation by proxmox ceph installer) puts her data?

Wanna be sure, that the pool with "replicated_rule" does not put data on nvmes aswell.

# rules
rule replicated_rule {
    id 0
    type replicated
    min_size 1
    max_size 10
    step take default
    step chooseleaf firstn 0 type host
    step emit
}
rule replicated_rule_nvme {
    id 1
    type replicated
    min_size 1
    max_size 10
    step take default class nvme
    step chooseleaf firstn 0 type host
    step emit
}

Does that mean default rule would still use all osds? So need to create two rules (one for hdd / one for nvme) and delete the replicated_rule?

 

[Alwin Antreich]

Does that mean default rule would still use all osds? So need to create two rules (one for hdd / one for nvme) and delete the replicated_rule?

Yes, exactly.

 

[jsterr]

Yes, exactly.

Thanks! It says: "Error EBUSY: crush ruleset replicated_rule 0 is in use" when I delete it. Might be cephfs? How can I find out where its used?
Edit: Ah webinterfaces says cephfs uses default replicated_rule I see. Figured it out, thanks.

Edit: Good to know: https://www.suse.com/support/kb/doc/?id=000019699
Better Link: https://pve.proxmox.com/pve-docs/chapter-pveceph.html#pve_ceph_device_classes

 

[Alwin Antreich]

Edit: Good to know: https://www.suse.com/support/kb/doc/?id=000019699

Yes, good to know.
https://pve.proxmox.com/pve-docs/chapter-pveceph.html#pve_ceph_device_classes

 

[jsterr]

Yes, good to know.
https://pve.proxmox.com/pve-docs/chapter-pveceph.html#pve_ceph_device_classes

Indeed but its missing howto change existing pools to use a new crush-rule

# ceph osd pool set $POOL_NAME crush_rule replicated_ssd

 

[Alwin Antreich]

Indeed but its missing howto change existing pools to use a new crush-rule

No, it isn't.

Once the rule is in the CRUSH map, you can tell a pool to use the ruleset.
ceph osd pool set <pool-name> crush_rule <rule-name>