Adding OSD to 4th Node

[vt600]

I have recently setup a three node cluster with Proxmox 3.2 and Ceph firefly, it looks to be running well. My problem is adding the 4th node. It looks as though I am unable to add OSDs to the new node since it is not a monitor. When I run pveceph createosd is get mon_command failed - command not known. Is there a different process to add the OSDs? Tried ceph-deploy but it was unable to find the config file.

Thanks for any help.

 

[hertog_jan]

Try adding it as a mon, then

 

[vt600]

I was hoping that there was a command I was missing. I didn't want to add and remove monitors everytime I need to add a disk or node.

 

[spirit]

Hi,

it should work without monitor on the 4th node (hopefully

some questions:

on the 4th node
----------------
- are you sure that you use firefly packages ?
- do you have the /etc/ceph.conf replicate from other proxmox nodes ?
- result of the command "#pveceph lspools" ?

 

[dietmar]

I guess I found a bug. Does the following patch helps?

diff --git a/PVE/API2/Ceph.pm b/PVE/API2/Ceph.pm
index 0fc6271..217b8ce 100644
--- a/PVE/API2/Ceph.pm
+++ b/PVE/API2/Ceph.pm
@@ -221,7 +221,7 @@ __PACKAGE__->register_method ({
        my $ceph_bootstrap_osd_keyring = PVE::CephTools::get_config('ceph_bootstrap_osd_keyring');
 
        if (! -f $ceph_bootstrap_osd_keyring) {
-           my $bindata = $rados->mon_command({ prefix => 'auth get client.bootstrap-osd', format => 'plain' });
+           my $bindata = $rados->mon_command({ prefix => 'auth get', entity => 'client.bootstrap-osd', format => 'plain' });
            PVE::Tools::file_set_contents($ceph_bootstrap_osd_keyring, $bindata);
        };

 

[vt600]

I will give the patch a shot and let you know.

Thanks

 

[vt600]

Please forgive my ignorance but I am missing something.
I copied the code and made it executable but it keeps failing.
I installed git because I thought it was needed but that didn't help either.

I am obviously doing something wrong. I have tried to find an article in the forum that outlines the steps but I can't find any.

 

[vt600]

Hi,

it should work without monitor on the 4th node (hopefully

some questions:

on the 4th node
----------------
- are you sure that you use firefly packages ?
- do you have the /etc/ceph.conf replicate from other proxmox nodes ?
- result of the command "#pveceph lspools" ?

I have checked the 4th node:
I ran pveceph install -version firefly when it was installed.
I checked the version and it is running 0.80.1
None of the nodes have ceph.conf in /etc they look to be located in /etc/pve which is on all the nodes.
pveceph lspools shows the pools I currently have

 

[vt600]

I couldn't figure out how to run the patch so i just changed the modified line in Ceph.pm and I am now able to create an OSD on the fourth node. Thank you very much.