Error when trying to del or create a ceph monitor

[Armando Ramos Roche]

Hi all..
I have a cluster whit 2 nodes working fine... until yesterday... and suddenly i see that 1 of these monitor not working, but manager is working.. and waiting for the monitor.
When try to delete this node i have a error:
root@pve2:/etc/apt# ceph mon remove pve2
mon.pve2 does not exist or has already been removed
and when try to create a new:
root@pve2:/etc/apt# pveceph mon create --mon-address 10.12.17.22
monitor 'pve2' already exists
So, what happen?? How solve this??
See the attached pic.
Thanks a lot

 

[Alwin]

I have a cluster whit 2 nodes working fine... until yesterday... and suddenly i see that 1 of these monitor not working, but manager is working.. and waiting for the monitor.

First off, Ceph needs at least three nodes for quorum.

root@pve2:/etc/apt# ceph mon remove pve2

There are two services on the host but none is running. And they might not be in the MON DB anymore. You will need to manually cleanup the service file and directory.

If the are not anymore visible in Ceph, then do something along this.

systemctl stop ceph-mon@<monid>.service
systemctl disable ceph-mon@<monid>.service
rm -r /var/lib/ceph/mon/ceph-<monid>/