Ceph MDS deployment?

[zdude]

I am trying to get an mds to start on proxmox 5.1 however none of the guides out there have been very helpful.

The cluster itself works fine, however an mds server will not start. I did install the mds server via

apt install ceph-mds

Spoiler: ceph.conf

[global]
auth_client_required = cephx
auth_cluster_required = cephx
auth_service_required = cephx
cluster_network = 10.0.0.0/24
fsid = ec4bbbab-90cc-42bd-ad88-634367074824
keyring = /etc/pve/priv/$cluster.$name.keyring
mon_allow_pool_delete = true
osd_journal_size = 5120
osd_pool_default_min_size = 2
osd_pool_default_size = 3
public_network = 10.0.0.0/24

[osd]
keyring = /var/lib/ceph/osd/ceph-$id/keyring

[mon.pvehost]
host = pvehost
mon_addr = 10.0.0.4:6789

[mds]
keyring = /var/lib/ceph/mds/ceph-$id/keyring

[mds.pvehost]
host = pvehost

Having deployed ceph on normal ubuntu and debian I am failing to understand what is so different about proxmox that breaks the mds server.


In order to activate the mds I was following the manual mds deployment guide on the ceph documentation site (forums won't let me link to the documentation).

 

[wolfgang]

Hi,

All needed services are installed you must only enable the service on a working cluster.

 

[MewBie]

Try this topic I wrote about installing CephFS.
https://forum.proxmox.com/threads/cephfs-installation.36459/

Edit: I have noticed problems on single node installations while playing around with it.
I tried it multiple times with 3 nodes and it worked every time, single nodes seem to be hit or miss. Mostly miss.

 

[zdude]

I was able to get it to work by installing ceph-deploy on a separate machine, hacking things to make it run (edit /etc/debian_releaseversion to 8.0 and run ceph-deploy gatherkeys). I was able to get an mds running that comes up on boot but cannot be started or stopped manually.

A Cephfs install is now running on an erasure coded data pool across 5 8tb drives with 3+1 encoding. (using bluestore OSDs)

 

[alexskysilk]

https://forum.proxmox.com/threads/proxmox-ve-5-0-beta2-released.34853/page-2#post-171734