Issues trying to delete mon

[cjdnad]

I recently rebuilt a VM that acts as a node in my cluster. I forgot to remove the monitor attached to it.

Now the new node is up and running with the same name and working fine I still have the mon in the gui as 'unknown'
Tried removing it from the command line but says it does not exist. Cannot see it on the conf file either if I pull up 'ceph quorum_status --format json-pretty'

BUT if I try to add a new on for the node it says - 'already exists'!!

Catch 22. How can I remove the trail of the old mon with the same name that does not exist but does when I try to create it.
The old one is dead as unknown and will not start.

Thanks

 

[rason]

Did you remove the mon dir in /var/lib/ceph?

 

[cjdnad]

Did you remove the mon dir in /var/lib/ceph?

Just to check - I need to remove the 'mon' in this directory?


root@pve3v:/var/lib/ceph# ls
bootstrap-mds bootstrap-mgr bootstrap-osd bootstrap-rbd bootstrap-rbd-mirror bootstrap-rgw crash mds mgr mon osd tmp
root@pve3v:/var/lib/ceph#

 

[rason]

Just to check - I need to remove the 'mon' in this directory?


root@pve3v:/var/lib/ceph# ls
bootstrap-mds bootstrap-mgr bootstrap-osd bootstrap-rbd bootstrap-rbd-mirror bootstrap-rgw crash mds mgr mon osd tmp
root@pve3v:/var/lib/ceph#

Sorry, I meant to only delete the contents of the directory related to the monitor you're trying to get rid of

 

[cjdnad]

Sorry, I meant to only delete the contents of the directory related to the monitor you're trying to get rid of

Sorry can you clarify the steps I should take?
pve3v is the new node currently working but there is the old mon pve3v hanging around that is unknown

 

[cjdnad]

sorry still stuck with the old mon, but cannot create a new one for the same node as it says it already exists, but cannot delete it says 'no such monitor'

 

[fiona]

Hi,
maybe this thread or one of the solutions in this thread helps?

 

[cjdnad]

Thanks. This did it:

ls /var/lib/ceph/mon #should see name of monitor
rm -rf /var/lib/ceph/mon/<name of monitor

systemctl stop ceph-mon@<impacted-mon>
systemctl disable ceph-mon@<impacted-mon>
systemctl daemon-reload
systemctl reset-failed
pveceph mon destroy <impacted-mon>

 

[cjdnad]

Not quite clean - monitor at least is not shown under monitors anymore - but still shown on the overview of services - will look at this another time when I have more time