[SOLVED] CephFS and pool device class selection

[lifeboy]

Creating a new MDS for a cephfs automatically creates a pool and storage class, but here it selects the wrong class.

We have two CRUSH rules:
ssd_rule, id 0
hdd_rule, id 1

CephFS gets created with id 0, but I want to have it use id 1. We can't create the cephfs_metadate and cephfs_data pool in advance, since ceph fs new cephfs metadata data then complains that pool 'cephfs_data' (id '5') has a non-CephFS application enabled.

How can the pool be select for CephFS?

 

[lifeboy]

I found that I can pre-create the pools and simply change the application. (It helps to read the man page! )

:~# ceph fs new cephfs cephfs_metadata cephfs_data
Error EINVAL:  pool 'cephfs_data' (id '9') has a non-CephFS application enabled.
:~# ceph osd pool application get cephfs_data
{
    "rbd": {}
}
:~# ceph osd pool application enable cephfs_data cephfs --yes-i-really-mean-it
enabled application 'cephfs' on pool 'cephfs_data'
:~# ceph osd pool application get cephfs_data
{
    "cephfs": {},
    "rbd": {}
}
:~# ceph osd pool application disable cephfs_data rbd --yes-i-really-mean-it
disable application 'rbd' on pool 'cephfs_data'
:~# ceph osd pool application get cephfs_data
{
    "cephfs": {}
}
:~# ceph osd pool application disable cephfs_metadata rbd --yes-i-really-mean-it
disable application 'rbd' on pool 'cephfs_metadata'
:~# ceph osd pool application enable cephfs_metadata cephfs --yes-i-really-mean-it
enabled application 'cephfs' on pool 'cephfs_metadata'
:~# ceph osd pool application get cephfs_metadata
{
    "cephfs": {}
}
:~# ceph fs new cephfs cephfs_metadata cephfs_data
new fs with metadata pool 10 and data pool 9

That did it.

 

[lifeboy]

There remains a little problem however. The pools we created manually now show as storage locations in the Proxmox GUI



In the above, cephfs_data and cephfs_metadata should not show up in this list. How do we tell them to not show here?

 

[Alwin]

In Datacenter -> Storage you should see the corresponding storage entries.

I found that I can pre-create the pools and simply change the application.

Yes, you can do that. Or specify the rule to use for the pool(s) afterwards.

 

[lifeboy]

In Datacenter -> Storage you should see the corresponding storage entries.


Yes, you can do that. Or specify the rule to use for the pool(s) afterwards.

How would I specify the rule to use for the pools afterwards?

 

[Alwin]

How would I specify the rule to use for the pools afterwards?

ceph osd pool set <pool-name> crush_rule <rule-name>
https://pve.proxmox.com/pve-docs/chapter-pveceph.html#pve_ceph_device_classes