Need some CEPH help here...
I have an existing Proxmox CEPH environment with the following:
5 nodes in a Proxmox environment (PVE v 4.4)
4 of the nodes have SSD based OSDs configured in a single RBD pool (32 total OSD's)
3 of those nodes are MON's
I've recently purchased 4 additional servers to increase the cluster's capacity.
Configured and added the 6th NEW node to the Proxmox cluster.
Installed and configured CEPH (the new node shows up in the CEPH OSD GUI page
Attempted to create the first OSD using the GUI....and this is where the trouble starts....
It appears that the OSD is successfully created and added to the existing cluster as the Total: under the CEPH page increases to 33....however the newly created OSD shows as DOWN and OUT. I noticed that the cluster then attempts to rebalance...
Another thing that seems to happen is if I run the following command from one of the nodes:
ceph tell osd.* version
....all 32 OSD's reply with their versions (all are reporting v 10.2.10) as expected but the NEW 33rd OSD reports the following error:
Error ENXIO: problem getting command descriptions from osd.32
osd.32: problem getting command descriptions from osd.32
Can someone explain what's happening here? I've googled all over the place and everything seems to indicate that a newly added OSD should show UP and IN...is this not the case? Will the new OSD show UP and IN AFTER the initial rebalance finishes?
Thanks!
<D>
I have an existing Proxmox CEPH environment with the following:
5 nodes in a Proxmox environment (PVE v 4.4)
4 of the nodes have SSD based OSDs configured in a single RBD pool (32 total OSD's)
3 of those nodes are MON's
I've recently purchased 4 additional servers to increase the cluster's capacity.
Configured and added the 6th NEW node to the Proxmox cluster.
Installed and configured CEPH (the new node shows up in the CEPH OSD GUI page
Attempted to create the first OSD using the GUI....and this is where the trouble starts....
It appears that the OSD is successfully created and added to the existing cluster as the Total: under the CEPH page increases to 33....however the newly created OSD shows as DOWN and OUT. I noticed that the cluster then attempts to rebalance...
Another thing that seems to happen is if I run the following command from one of the nodes:
ceph tell osd.* version
....all 32 OSD's reply with their versions (all are reporting v 10.2.10) as expected but the NEW 33rd OSD reports the following error:
Error ENXIO: problem getting command descriptions from osd.32
osd.32: problem getting command descriptions from osd.32
Can someone explain what's happening here? I've googled all over the place and everything seems to indicate that a newly added OSD should show UP and IN...is this not the case? Will the new OSD show UP and IN AFTER the initial rebalance finishes?
Thanks!
<D>