mount cephfs into a lxc

[KeyzerSuze]

Hi
I have a 5 node proxmox cluster, I created a ceph cluster from those nodes and a cephfs
I would like to access the ceph fs from other devices in my home lab. I thought I would create a lxc (debian12) with nfs and then rbind the cephfs from the underlying node into the lxc

so my lxc is privileged + nested + cifs so that I can run nfs-kernel

so I thought I have 2 options
a) rbind - how do I do that say my cephfs is mount /mnt/pve/cephfs on the nodes and I have created /mnt/pve/cephfs/hme1/fs/ and I want to mount that in my lxc as /srv/hme1fs
b) I can mount the cephfs natively into the lxc - i believe install ceph-common. but I need some sort of user and secret key where do I find these

option a seems the easiest way - does it work ?
I wouldn't mind doing option b because I could set this up on my other devices as well and by pass nfs so a learning experience

thought welcome

 

[aaron]

Option A with a bind mount should definitely work: https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_bind_mount_points

But doesn't CephFS come with a NFS Ganesha server? https://docs.ceph.com/en/latest/cephfs/nfs/
That might work, or, alternatively, depending on your clients, mount CephFS directly in the clients. For this option you should ideally have a client ID and key per client. https://docs.ceph.com/en/latest/cephfs/client-auth/

 

[alexskysilk]

Option A with a bind mount should definitely work

Can confirm

But doesn't CephFS come with a NFS Ganesha server?

Not in the PVE implementation at least not OOB. maybe you want to add it? more seriously, this have little purpose. Any nfs capable client would be better served by attaching cephfs directly (Windows can and should use smb.)

 

[aaron]

at least not OOB

All Ceph packages should be available to install, even with the Proxmox Ceph repos .

Some things might not work, but that is in many cases because Ceph deployed on Proxmox VE is not using the Ceph Orchestra method, which prohibits a few things that require it.

Any nfs capable client would be better served by attaching cephfs directly (Windows can and should use smb.)

There is also a Windows client for CephFS and I know at least one customer who is using their Ceph cluster primarily as CephFS network share with Windows clients

 

[KeyzerSuze]

thanks every one for the input so what i have gathered is
ceph comes with a NFS server => NFS Ganesha server nice to know - when i get around to make a seperate ceph cluster i will look at that.
But hesitant to stuff up the ceph install under PVE I guess becuase its linked.

so a bind mount would work to a lxc which is doing my FS export - so that could be nfs or cifs if I wanted.

But I presume to get the best out of this it would be better to learn to mount a ceph fs onto a server, seem like you have provide some links for that as well << think this is the path i will try first

Thanks again

 

[KeyzerSuze]

F

thanks every one for the input so what i have gathered is
ceph comes with a NFS server => NFS Ganesha server nice to know - when i get around to make a seperate ceph cluster i will look at that.
But hesitant to stuff up the ceph install under PVE I guess becuase its linked.

so a bind mount would work to a lxc which is doing my FS export - so that could be nfs or cifs if I wanted.

But I presume to get the best out of this it would be better to learn to mount a ceph fs onto a server, seem like you have provide some links for that as well << think this is the path i will try first

Thanks again

or those that follow on from this


ceph osd pool ls
.mgr
cephPool01
cephfs_data
cephfs_metadata

not sure how to get namespaces from the cli - but using the proxmox webui i can have cephfs


# create user with no auth
ceph auth get-or-create-key client.max

# modify auth I think this allows me to mount the cephfs I think
ceph auth caps client.max mon 'allow r' osd 'allow rw pool=cephPool01 namespace=cephfs'