[SOLVED] Remnants of ceph configuration after pveceph purge?

[fartboner]

I'm new to proxmox and ceph, and I messed up a cluster I have. I didn't have anything critical so I decided I'd start fresh. My cluster was unhealthy so I couldn't gracefully turn everything down so I assumed I'd need to purge. I read a few threads that mention how to purge ceph and I followed them. The ceph installation wizard within the web GUI gives me the option to reinstall but I'm running into an error on the configuration tab on the web interface after installation.

"Could not connect to ceph cluster despite configured monitors (500)"

Where can I find where the remnants are left over so that I can have a clean reinstall?


EDIT:
I rebooted after running my purge script that I compiled from the different threads and it looks like I'm able to finish via the wizard. False alarm!

 

[Philipp Hufnagl]

Can you tell me in more detail how you purged Ceph?

 

[fartboner]

Can you tell me in more detail how you purged Ceph?

Because nothing was responding, I SSH'd into each host and ran this script:

#!/bin/bash
systemctl restart pvestatd
rm -rf /etc/systemd/system/ceph*
killall -9 ceph-mon ceph-mgr ceph-mds
umount /var/lib/ceph/osd/ceph*
rm -rf /etc/ceph /etc/pve/ceph.conf /etc/pve/priv/ceph* /var/lib/ceph
pveceph purge
systemctl restart pvestatd
apt purge ceph-mon ceph-osd ceph-mds -y
systemctl restart pvestatd
rm /etc/init.d/ceph
# my additions
mkdir /etc/ceph
mkdir /var/lib/ceph/
mkdir /var/lib/ceph/bootstrap-osd
mkdir /var/lib/ceph/mgr

After that, I was able to reinstall ceph but ran into that monitor issue. After a reboot that seems to be resolved, though.

 

[Philipp Hufnagl]

did you run that script on all nodes?
Can you give me the output of journalctl -b?

 

[fartboner]

I did. Tried to use the codeblock but it was too long. See attached.

 

[fartboner]

I'm intermittently receiving "got timeout (500)" messages when making changes or viewing the ceph section of the hosts. Is that something to be concerned about?

 

[Philipp Hufnagl]

try

• removing the mkdir additions of you from the script
• run the script
• run on the terminalapt reinstall ceph ceph-common

 

[fartboner]

try

• removing the mkdir additions of you from the script
• run the script
• run on the terminalapt reinstall ceph ceph-common

I'm currently reinstalling ceph on the remaining nodes, and setting up OSDs. I think that script would destroy the cluster again? Or is this to resolve the "got timeout (500)" messages?

 

[Philipp Hufnagl]

are you still getting the monitor error?

 

[fartboner]

No. I edited my post shortly after posting. Sorry for wasting your time

 

[Philipp Hufnagl]

Oh.

Well glad it works now

If you are concerned with got timeout messages you can journalctl -f and see if something strange pops up

 

[fartboner]

Appreciate the guidance!