Adding OSD to 4th Node

vt600

New Member
May 21, 2014
7
0
1
I have recently setup a three node cluster with Proxmox 3.2 and Ceph firefly, it looks to be running well. My problem is adding the 4th node. It looks as though I am unable to add OSDs to the new node since it is not a monitor. When I run pveceph createosd is get mon_command failed - command not known. Is there a different process to add the OSDs? Tried ceph-deploy but it was unable to find the config file.

Thanks for any help.
 
I was hoping that there was a command I was missing. I didn't want to add and remove monitors everytime I need to add a disk or node.
 
Hi,

it should work without monitor on the 4th node (hopefully ;)

some questions:

on the 4th node
----------------
- are you sure that you use firefly packages ?
- do you have the /etc/ceph.conf replicate from other proxmox nodes ?
- result of the command "#pveceph lspools" ?
 
I guess I found a bug. Does the following patch helps?

Code:
diff --git a/PVE/API2/Ceph.pm b/PVE/API2/Ceph.pm
index 0fc6271..217b8ce 100644
--- a/PVE/API2/Ceph.pm
+++ b/PVE/API2/Ceph.pm
@@ -221,7 +221,7 @@ __PACKAGE__->register_method ({
        my $ceph_bootstrap_osd_keyring = PVE::CephTools::get_config('ceph_bootstrap_osd_keyring');
 
        if (! -f $ceph_bootstrap_osd_keyring) {
-           my $bindata = $rados->mon_command({ prefix => 'auth get client.bootstrap-osd', format => 'plain' });
+           my $bindata = $rados->mon_command({ prefix => 'auth get', entity => 'client.bootstrap-osd', format => 'plain' });
            PVE::Tools::file_set_contents($ceph_bootstrap_osd_keyring, $bindata);
        };
 
Please forgive my ignorance but I am missing something.
I copied the code and made it executable but it keeps failing.
I installed git because I thought it was needed but that didn't help either.

I am obviously doing something wrong. I have tried to find an article in the forum that outlines the steps but I can't find any.
 
Hi,

it should work without monitor on the 4th node (hopefully ;)

some questions:

on the 4th node
----------------
- are you sure that you use firefly packages ?
- do you have the /etc/ceph.conf replicate from other proxmox nodes ?
- result of the command "#pveceph lspools" ?

I have checked the 4th node:
I ran pveceph install -version firefly when it was installed.
I checked the version and it is running 0.80.1
None of the nodes have ceph.conf in /etc they look to be located in /etc/pve which is on all the nodes.
pveceph lspools shows the pools I currently have
 
I couldn't figure out how to run the patch so i just changed the modified line in Ceph.pm and I am now able to create an OSD on the fourth node. Thank you very much.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!