Creating and Mounting CephFS on 4.2 - HowTo

[mjoconr]

Hi All
I would like to use CephFS for ISO files and the initial location for Backups before rsyncing them to a remote location.
The issue I can not seem to get the steps worked out to actually create the file system in the ceph storage.

Has anyone written a howto or just listed the steps required ? I get to the point of creating teh MDS and have key issues.

I'm sure there will be other issue after this.

root@blade1:/etc/ceph# ceph-deploy mds create blade1
[ceph_deploy.conf][DEBUG ] found configuration file at: /root/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.28): /usr/local/bin/ceph-deploy mds create blade1
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] subcommand : create
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x7faff600f5f0>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] func : <function mds at 0x7faff6469b90>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] mds : [('blade1', 'blade1')]
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.mds][DEBUG ] Deploying mds, cluster ceph hosts blade1:blade1
[ceph_deploy][ERROR ] RuntimeError: bootstrap-mds keyring not found; run 'gatherkeys'

Any help would be greatly appreciated.
Mike

 

[Melanxolik]

why don't you use glusterfs for storage iso?
it's more usefull then ceph.

 

[mjoconr]

Because that would require me taking the whole production system offline. You need to justify you statement about glusterfs.
I actually think sheepdog would be better than both as its got a lot less moving parts and supports NBD, iscsi, fuseFS and HTTP out of the box.

 

[alexskysilk]

It is possible to set up MDS but its not a good solution. The simplest and easiest way to do what you ask is to create a NAS vm.

 

[PigLover]

I don't think I'd trust CephFS for much until you are running Jewel. Before that is was quite unstable and didn't have complete recovery tools (file system check/recovery). Since proxmox 4.2 is still at Hammer - unless you've done something to upgrade it - you are wise not do do this yet.

As @alexskysilk suggests, creating a NAS VM that uses Ceph Image files as its "disks" is probably your safe bet.

 

[alexskysilk]

Even once Proxmox incorporates Jewel, there are still considerations that make commingling Ceph RBD and FS a bad idea; bear in mind that there is hashing overhead for both functions, which will put your nodes' CPU and ram into contention. Considering the use case this is just not an effective way to use your resources.

 

[PigLover]

Fair play to that, but at least when Jewel is around you get to make those kind of engineering trade-offs in how you deploy. You and I might not agree with peoples choices, but depending on their situation it might make sense.

For now, prior to Jewel being available under Proxmox, CephFS is just to unstable to even consider using. When it doesn't actually work there is no trade-off to be made.

 

[mjoconr]

Still given all that I can not take the production system online and the MDS is something which can be turned on and give me the storage I need. a VM NAS would not be running at boot time meaning the real servers would have trouble at boot time. ie mounting the NFS storage for ISOs and backup area during boot.
So my question still stands does anyone have the steps I need to take to get MDS and CEPHFS working with Proxmox 4.2