[SOLVED] Mount a cephfs to a vm

[Donovan Hoare]

Good day all.
Sorry for all the noob questions.

I have ceph running on my 6node proxmox cluster.
After some other noob questions ive asked over the last few days. its running nicely.
I would like to now add a cephfs share to a vm
Assumption : when i add to a vm, it woule be like mounting a NFS share so when i write to that mount point, all servers with the same mount point can see the data.

So what i have done.
I addeded 3 metatdata servers (don't know why i did 3 but everything in Ceph seems to like 3).
of which now one is active and 2 are in standby. see image attached.

i then added one cephFS, which is what im assuming is the sharename. here iredmailstore




Now on a VM i have spun up.
A added a network interface on the bridge of the Ceph client. it can ping the 10.0.45.0-24 range.
I then installed

apt install ceph-common

on the server i created a key file then moved to the vm.

ceph auth get-key client.admin > cephfs.secret

i then scp'ed the file to the vm

On the vm i did

mkdir -p /mnt/iredmailstore
mount -t ceph 10.0.45.24,10.0.45.25,10.0.45.26:/ /mnt/iredmailstore -o name=root,secretfile=/root/iredmailstore.secret,fs=iredmailstore

However this is not working im getting this output

mount -t ceph 10.0.45.24:6789:/ /mnt/iredmailstore -o name=admin,secretfile=/root/iredmailstore.secret,fs=iredmailstore
did not load config file, using default settings.
2024-05-23T16:20:54.580+0000 7f3fcc53cf40 -1 Errors while parsing config file!

2024-05-23T16:20:54.580+0000 7f3fcc53cf40 -1 can't open ceph.conf: (2) No such file or directory

2024-05-23T16:20:54.580+0000 7f3fcc53cf40 -1 Errors while parsing config file!

2024-05-23T16:20:54.580+0000 7f3fcc53cf40 -1 can't open ceph.conf: (2) No such file or directory

unable to get monitor info from DNS SRV with service name: ceph-mon
2024-05-23T16:20:54.592+0000 7f3fcc53cf40 -1 failed for service _ceph-mon._tcp

2024-05-23T16:20:54.592+0000 7f3fcc53cf40 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory

mount error: no mds server is up or the cluster is laggy
mount error: no mds server is up or the cluster is laggy

Any help would be appriciated.

Regards

 

[Donovan Hoare]

OK i found what i was doing wrong.
When mounting i was trying to connect to the ip address of the metadata servers.

However when i changed it to one of the ipaddresses of the monitor servers. it worked.

 

[arunas]

for me this didn't worked. PVE 8.2.7, 3 node cluster.

I got error

mount error 22 = Invalid argument

I had to remove secretfile from options and change name from root to admin, because ceph keyring named /etc/pve/priv/ceph.client.admin.keyring

mount.ceph 172.x.x.1,172.x.x.2,172.x.x.3:/ /mnt/cephfs/nfs -o name=admin,fs=cephfs

if you leave option "name=roo" ir will be looking for /etc/pve/priv/ceph.client.root.keyring