[SOLVED] RBD error when creating VM using CEPH

[mjoconr]

Hi All

I'm having trouble creating VM's using RBD over CEPH, I've tried a number of methods to create the storage.
1. have spaces between the ip addresses
2. have a comma or a semicolon
3. all of the above using <ip>:6789

I always get the following error
---
TASK ERROR: create failed - rbd error: rbd: couldn't connect to the cluster!
---
I've checked the iptables and there are no rules at all yet.

From the CLI I was able to create a block image and then mount it so I'm sure the cluster is working correctly.

I've not been able to find any logs which a better description of the error.
Mike

 

[proxtest]

Maybe u forgot
?
https://pve.proxmox.com/wiki/Ceph_Server

You also need to copy the keyring to a predefined location.

Note that the file name needs to be storage id + .keyring . storage id is the word after this in /etc/pve/storage.cfg : 'rbd:'
# cd /etc/pve/priv/
# mkdir ceph
# cp /etc/ceph/ceph.client.admin.keyring ceph/my-ceph-storage.keyring

 

[mjoconr]

Ok, yes I was missing those files. We are creating the storage via the GUI why does it not do this ?

Still I've done this, but still getting the error.

rbd: lxc
monhost 10.1.5.201;10.1.5.203;10.1.5.205;10.1.5.207
username admin
content rootdir
pool rbd
krbd

root@blade1:~# ls /etc/pve/priv/ceph
kvm.keyring lxc.keyring
root@blade1:~#


But the problem continues.

 

[mjoconr]

Ok so I found an error in a log file which would seem to be the issue, still not sure how to fix
2016-05-23 10:58:42.209426 7fcfc2903700 0 cephx server client.admin: unexpected key: req.key=2d9c8444e2560735 expected_key=7a67648ddf3f5a52
2016-05-23 10:58:42.248380 7fcfc2903700 0 cephx server client.admin: unexpected key: req.key=7b9edb10ebb149b9 expected_key=eb91c78e730fd2e
2016-05-23 10:58:42.677177 7fcfc2903700 0 cephx server client.admin: unexpected key: req.key=363e09ee7888f323 expected_key=f2b4d39d915d90f4
2016-05-23 10:58:42.712069 7fcfc2903700 0 cephx server client.admin: unexpected key: req.key=a1dd56cb7f16d7b8 expected_key=7cdc8205da8aaf01
2016-05-23 10:58:42.754009 7fcfc2903700 0 cephx server client.admin: unexpected key: req.key=cf9afaf3f6fbde03 expected_key=85816b10ba55a3cf
2016-05-23 10:58:42.789018 7fcfc2903700 0 cephx server client.admin: unexpected key: req.key=5aadbc87a3a863e5 expected_key=2de2c8052c4ec6d1

Mike

 

[mjoconr]

Ok so this is how I fixed it
ceph auth list give me a different key than the one on my priv/ceph files so I changed the key to match the one given in the list and all fixed I have no idea why the key I was copying was different

Mike