Ceph config broken

[joshbgosh10592]

ceph-mon -i <ID> --mkfs

You can recreate the DB with the above command.

Thank you, tried that and now:

root@PVE-2:~# ceph-mon -i PVE-2 --mkfs
2019-06-28 11:00:25.671562 7f0e2cc90100 -1 mon.PVE-2@-1(probing) e0 unable to find a keyring file on /etc/pve/priv/ceph.mon.PVE-2.keyring: (2) No such file or directory
2019-06-28 11:00:25.671607 7f0e2cc90100 -1 ceph-mon: error creating monfs: (2) No such file or directory

I did some reading and found the command "ceph versions" of which fails to run on any of my nodes...

root@PVE-1:~# ceph versions
2019-06-28 11:07:35.817570 7fde233b0700  0 monclient(hunting): authenticate timed out after 300
2019-06-28 11:07:35.817617 7fde233b0700  0 librados: client.admin authentication error (110) Connection timed out
[errno 110] error connecting to the cluster

yet, if I try to install:

root@PVE-1:~# pveceph install
update available package list
start installation
Reading package lists... Done
Building dependency tree       
Reading state information... Done
gdisk is already the newest version (1.0.1-1).
ceph-fuse is already the newest version (12.2.12-pve1).
ceph-mds is already the newest version (12.2.12-pve1).
ceph-common is already the newest version (12.2.12-pve1).
ceph is already the newest version (12.2.12-pve1).
0 upgraded, 0 newly installed, 0 to remove and 34 not upgraded.

installed ceph luminous successfully

Would the fix be something as easy as "pveceph uninstall"? Or is there just an easier way to completely remove all Ceph configs, filesystems, and then reinstall it? Basically, there's nothing in my Ceph storage, I just would very much rather not blow up the VMs on my Proxmox nodes using their own local storage.

 

[Alwin]

If no service is running anymore, then you need to remove all installed ceph packages (apt remove <packages>) and then could do a 'pveceph purge'.

 

[joshbgosh10592]

Thank you!
So, I ran

apt remove ceph ceph-base ceph-mon ceph-osd

that seems to have worked fine, but pveceph purge only completed silently on one node (PVE-Witness)
The other two returned:

root@PVE-1:~# pveceph purge
unable to get monitor info from DNS SRV with service name: ceph-mon
no monitors specified to connect to.

Not sure what to do about that? The WebApp at least shows that the packages aren't installed for any of the nodes, so that's good.

 

[Alwin]

Not sure what to do about that? The WebApp at least shows that the packages aren't installed for any of the nodes, so that's good.

The pveceph purge does some checking before removal. You can just remove the ceph config and the '/var/lib/ceph' folder by hand too.

 

[joshbgosh10592]

The pveceph purge does some checking before removal. You can just remove the ceph config and the '/var/lib/ceph' folder by hand too.

What directories/files do I need to remove to remove the ceph config?

 

[joshbgosh10592]

Or is the ceph config in that directory? I've only found: http://docs.ceph.com/docs/master/rados/deployment/ceph-deploy-purge/ online.