I am trying to set up active MDS daemon on ceph cluster with three Nodes.

Status
Not open for further replies.

maxellju

New Member
May 2, 2022
2
0
1
Hello community ,

I had set up a Ceph cluster with three Nodes. The MDS daemon is not comming up



What I had set up


ceph auth get-or-create mds.${3} mon 'profile mds' mgr 'profile mds' mds 'allow *' osd 'allow *' > /var/lib/ceph/mds/ceph-${3}/keyring


# create filesystem pools
ceph osd pool create cephfs0_data
ceph osd pool create cephfs0_metadata

# create filesystem with explicit pools
ceph fs new cephfs0 cephfs0_metadata cephfs0_da




My environment

[root@node3 mnt]# ceph -s
cluster:
id: 1a15bbb4-b62c-441b-889b-4e5a230911d1
health: HEALTH_ERR
1 pools have many more objects per pg than average
2 filesystems are offline
2 filesystems are online with fewer MDS than max_mds
2 daemons have recently crashed

services:
mon: 3 daemons, quorum node1,node2,node3 (age 2h)
mgr: node3(active, since 2h), standbys: node2, node1
mds: cephfs:1 cephfs-3:0
osd: 108 osds: 108 up (since 2h), 108 in (since 8M)
rgw: 1 daemon active (node1.rgw0)

task status:

data:
pools: 28 pools, 817 pgs
objects: 102.46k objects, 36 GiB
usage: 237 GiB used, 982 TiB / 982 TiB avail
pgs: 817 active+clean




The are the two errors I am getting are :


MDS_ALL_DOWN: 2 filesystems are offline

MANY_OBJECTS_PER_PG: 1 pools have many more objects per pg than average

MDS_UP_LESS_THAN_MAX: 2 filesystems are online with fewer MDS than max_mds

RECENT_CRASH: 2 daemons have recently crashed


[root@node3 mds]# ceph-mds --hot-standby 3
2022-05-04T02:30:58.508-0700 7f3ed0088600 -1 auth: error reading file: /var/lib/ceph/mds/ceph-admin/keyring: bufferlist::read_file(/var/lib/ceph/mds/ceph-admin/keyring): read error:(21) Is a directory
2022-05-04T02:30:58.508-0700 7f3ed0088600 -1 auth: failed to load /var/lib/ceph/mds/ceph-admin/keyring: (21) Is a directory
2022-05-04T02:30:58.508-0700 7f3ed0088600 -1 auth: error reading file: /var/lib/ceph/mds/ceph-admin/keyring: bufferlist::read_file(/var/lib/ceph/mds/ceph-admin/keyring): read error:(21) Is a directory
2022-05-04T02:30:58.508-0700 7f3ed0088600 -1 auth: failed to load /var/lib/ceph/mds/ceph-admin/keyring: (21) Is a directory
2022-05-04T02:30:58.508-0700 7f3ed0088600 -1 auth: error reading file: /var/lib/ceph/mds/ceph-admin/keyring: bufferlist::read_file(/var/lib/ceph/mds/ceph-admin/keyring): read error:(21) Is a directory
2022-05-04T02:30:58.508-0700 7f3ed0088600 -1 auth: failed to load /var/lib/ceph/mds/ceph-admin/keyring: (21) Is a directory
2022-05-04T02:30:58.508-0700 7f3ed0088600 -1 monclient: keyring not found
failed to fetch mon config (--no-mon-config to skip)


Could you help please. Thanks
 
Status
Not open for further replies.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!