ceph removed node, but osd's still show up in pve.

[RobFantini]

Hello

I removed a node from pve/ceph cluster.

but the osd's still show in pve when at Ceph/OSD . status for each osd is 'down/out'

The node itself does not show up on pve or cli ' pvecm nodes '

I Tried removing in pve by selecting the osd and pressing 'Remove' , the result is : 'Connection error 595: No route to host '
And checked for config files, but none of the text ones seem to have osd's defined.

is it possible to remove the osd's ?

PS: Is there a correct way to remove a ceph system from cluster? Let me know I'll post it to wiki.

 

[dietmar]

The correct way would be to remove/delete all ceph resources running on that node before removing the node.

Manual commands are described here:

http://ceph.com/docs/master/rados/operations/add-or-rm-osds/

So you have to do something like:

# ceph osd out {osd-num}
# ceph osd crush remove osd.{osd-num}
# ceph auth del osd.{osd-num}
# ceph osd rm {osd-num}