Proxmox 5.0/Ceph luminus "rbd error: rbd: couldn't connect to the cluster!"

[Spiros Pap]

Hi all,

I'm getting the error: "rbd error: rbd: couldn't connect to the cluster!" when i create a VM with rbd storage from GUI.
I have installed the client keyring in /etc/pve/priv/ceph/my-ceph-storage.keyring.
# rbd ls -p images #returns no error. The "images" is the pool I have created.
#

cat /etc/pve/storage.cfg
....
rbd: rdbvms
monhost 10.102.247.145 10.102.247.146 10.102.247.147
content rootdir,images
krbd 0
pool images
username admin

I have also tried to create the ceph storage with:
monhost 10.102.247.145;10.102.247.146;10.102.247.147 (semicolon instead of spaces)
or with port numbers

root@pve1:~# ceph -s
cluster:
id: a7c8f4a9-daa3-413c-8b43-6b1067c4a18e
health: HEALTH_OK

services:
mon: 3 daemons, quorum 0,1,2
mgr: 0(active), standbys: 1, 2
osd: 12 osds: 12 up, 12 in

data:
pools: 1 pools, 512 pgs
objects: 0 objects, 0 bytes
usage: 12803 MB used, 382 GB / 394 GB avail
pgs: 512 active+clean


Any ideas? Any tests I can do to find the issue?

Thanx,
Spiros

 

[dcsapak]

I have installed the client keyring in /etc/pve/priv/ceph/my-ceph-storage.keyring.
# rbd ls -p images #returns no error. The "images" is the pool I have created.
#

cat /etc/pve/storage.cfg
....
rbd: rdbvms

the file has to have the name of the storage
e.g. in your case:
/etc/pve/priv/ceph/rbdvms.keyring

 

[Spiros Pap]

the file has to have the name of the storage
e.g. in your case:
/etc/pve/priv/ceph/rbdvms.keyring

Worked! thanx! Please make it more clear in the docs.

sp

 

[dcsapak]

Worked! thanx! Please make it more clear in the docs.

great that you fixed it

but it already is quite clear in the docs:

Note that the file name needs to be storage id + .keyring storage id is the expression after 'rbd:' in /etc/pve/storage.cfg which is my-ceph-storage in the current example.

do you have a suggestion how to improve it?

 

[Spiros Pap]

great that you fixed it

but it already is quite clear in the docs:



do you have a suggestion how to improve it?

Not actually! I did not actually see the note.. I guess i missed it because it has much text I just tried the example and it worked, because "ceph.client.admin.keyring" actually exists and i thought "if i need not change the source, why change the destination? ". Perhaps before the note give the idea like:
cp /etc/pve/priv/ceph.client.admin.keyring /etc/pve/priv/ceph/{my-ceph-storage}.keyring
This way, copy/paste will fail and you will have to read the note...

A humans soul is an abyss...

cheers,
Sp