Ceph, MDS, CephFS questions

[Vassilis Kipouros]

Good evening,

Been trying to deploy MDS & CephFS by compiling a couple of very scarce threads from here & there.

But no luck till now. Ceph-deploy tool does not work as expected and couldn't
start the MDS after manually adding the [mds] section etc in ceph.conf

Is MDS and CephFS management in Proxmox a feature we should be expecting soon?

Has anyone successfully deployed CephFS on Proxmox that can give me/us some advice?

* I have a small 3 node cluster with 5x1T OSD's each and my ultimate plan is to deploy CephFS on an erasure coded pool that will act like raid5 between the three nodes, being able to sustain a whole node failure.

** Currently I have replicated pool of two copies, which gives me about 7T of storage, but the data I need to fit in that pool is 9T that's why I need the erasure code

*** Another question: Say all your nodes get fried but you magically manage to save your OSD's.
Is there a way to add/import those OSD's on another fresh/new (or not) cluster and maintain your data?


Kind regards...

 

[mackerman]

I also am interested in MDS on CephFS, to provide NFS file shares from an even smaller cluster. Not having ceph-deploy seems intentional as it does not integrate as well with the pve cluster. Eventually I will also try building a MDS manually per Ceph's documentation.

Still new to Ceph, but I don't think the recovery procedures are optimized for reimporting OSDs. Very large clusters will lose nodes all the time, and just have to add new ones faster than they fail. Perhaps backup RBD or other objects, and save the existing OSD import as a last-chance effort.

 

[Vassilis Kipouros]

This is so F#$% frustrating...

ceph-deploy purge, removes proxmox packages also... FFS

 

[fabian]

This is so F#$% frustrating...

ceph-deploy purge, removes proxmox packages also... FFS

yes, because we need librbd for qemu. you should be able to get one (or multiple) MDS instances running using http://docs.ceph.com/docs/master/install/manual-deployment/#adding-mds

 

[AlexLup]

+1 for ceph MDS support, I managed to get it up and running once but am back to errors!