Sharing CephFS among VMs with a size limit

[etakmit]

Three node cluster
Running Ceph
5 OSDs per node - 15 total

I can create a cephFS and use it for storage for VMs, ISOs, etc no problem.

What I want to do is have shared mounted storage for VMs via the ceph cluster.

i.e. I create a kubernetes cluster and have /kube be shared storage for all of the drives - but I want it to be limited in size (i.e. my cephFS is 7TB - but i want this share to be 500GB)

I haven't come across a clean way to do so yet. Maybe I'm barking up the wrong tree as well.

 

[William Edwards]

Can you be more specific about the issues/doubts you're facing?

 

[etakmit]

Can you be more specific about the issues/doubts you're facing?

If I create a cephFS - it takes the entire ceph cluster size (let's say its 5TB)

then I can create a directory (kubedirectory) on the cephfs and share that via virtiofs or the like

but the problem is that directory will still encompass the whole size of the cephfs and I'd like to only give VMs what I need to give them

(i.e a mountable drive/directory that is space limited)

 

[William Edwards]

If I create a cephFS - it takes the entire ceph cluster size (let's say its 5TB)

then I can create a directory (kubedirectory) on the cephfs and share that via virtiofs or the like

but the problem is that directory will still encompass the whole size of the cephfs and I'd like to only give VMs what I need to give them

(i.e a mountable drive/directory that is space limited)

https://docs.ceph.com/en/latest/cephfs/quota/