CEPH Configuration: CephFS only on a Specific Pool?

Illydth

New Member
Aug 23, 2022
8
2
1
So hopefully this isn't a duplicate of something well asked:

I have 2 pools of disks: The default Ceph pool that I use for VMs (the large pool on the larger disks) and a smaller pool I intend to use as K8s storage. The larger pool I want to keep Exclusively for VM storage, but the smaller pool I want to put CephFS on top of.

And that leads to the basic question: How do I specify the pool that Ceph is going to put data into? I tried creating a "cephfs_data" pool on the secondary area and putting the cephfs_metadata pool on primary, and then using:

ceph fs new cephfs cephfs_metadata cephfs_data

But it returned: "Error EINVAL: poll 'cephfs_data' (id '11') has a non-CephFS application enabled."

I assume because I created it through the normal "pool" creation in the GUI it's giving me this.

Anyone know how to create the CephFS pool on something other than the "Default" Ceph storage?
 
realize this is a late answer, but ran into this thread as I prepare to re-pool my metadata.

Short answer: Each pool must be "enabled" for specific applications. When you create the pool via the UI or command line, it defaults to RBD. You can see which applications are enabled for your pool like so:


Code:
root@pmx2:~# ceph osd pool application get cephfs_metadata
{
    "cephfs": {
        "metadata": "cephfs"
    }
}

root@pmx2:~# ceph osd pool application get cephfs_data
{
    "cephfs": {
        "data": "cephfs"
    }
}

root@pmx2:~# ceph osd pool application get SM32
{
    "rbd": {}
}


Following that model, we'd need to enable the target pool for cephfs:

Code:
ceph osd pool application enable {pool-name} {application-name}

or

ceph osd pool application set --pool=poolname --app=string  --key=string --value=string


(will update this thread more after I finish migrating pools)

REFERENCE:
* https://docs.ceph.com/en/quincy/api/mon_command_api/#osd-pool-application-get
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!