CEPH Configuration: CephFS only on a Specific Pool?

[Illydth]

So hopefully this isn't a duplicate of something well asked:

I have 2 pools of disks: The default Ceph pool that I use for VMs (the large pool on the larger disks) and a smaller pool I intend to use as K8s storage. The larger pool I want to keep Exclusively for VM storage, but the smaller pool I want to put CephFS on top of.

And that leads to the basic question: How do I specify the pool that Ceph is going to put data into? I tried creating a "cephfs_data" pool on the secondary area and putting the cephfs_metadata pool on primary, and then using:

ceph fs new cephfs cephfs_metadata cephfs_data

But it returned: "Error EINVAL: poll 'cephfs_data' (id '11') has a non-CephFS application enabled."

I assume because I created it through the normal "pool" creation in the GUI it's giving me this.

Anyone know how to create the CephFS pool on something other than the "Default" Ceph storage?

 

[dlasher]

realize this is a late answer, but ran into this thread as I prepare to re-pool my metadata.

Short answer: Each pool must be "enabled" for specific applications. When you create the pool via the UI or command line, it defaults to RBD. You can see which applications are enabled for your pool like so:

root@pmx2:~# ceph osd pool application get cephfs_metadata
{
    "cephfs": {
        "metadata": "cephfs"
    }
}

root@pmx2:~# ceph osd pool application get cephfs_data
{
    "cephfs": {
        "data": "cephfs"
    }
}

root@pmx2:~# ceph osd pool application get SM32
{
    "rbd": {}
}

Following that model, we'd need to enable the target pool for cephfs:

ceph osd pool application enable {pool-name} {application-name}

or

ceph osd pool application set --pool=poolname --app=string  --key=string --value=string

(will update this thread more after I finish migrating pools)

REFERENCE:
* https://docs.ceph.com/en/quincy/api/mon_command_api/#osd-pool-application-get