pveceph purge wipes entire ceph cluster, not just a host.

[AngryAdm]

Yeah, one of those proxmox adventures where shit just hits the fan for absolutely no obvious reason.

Please Add to the help file the definition of what ceph related data and configuration files and where, because it is not local.. I am very very very anoyed!



USAGE: pveceph purge [OPTIONS]

Destroy ceph related data and configuration files.

-crash <boolean>

Additionally purge Ceph crash logs, /var/lib/ceph/crash.

-logs <boolean>

Additionally purge Ceph logs, /var/log/ceph.


So, how do I get the OSD's back into a cluster again?
Not possible is not the correct answer!
There are 0 managers, 0 mons 0 SSD's just "configure ceph" on node 1-6.
pveceph purge was run on node 6... Why in gods name is the config being wiped on the other nodes?!?!?

 

[AngryAdm]

Oh and upon configuring the cluster... loads of errors. Reinstall proxmox for the f..... 10th time!
Proxmox 7 -> Proxmox ME
Proxmox Vista is next?

 

[AngryAdm]

Jep, proxmox install screwed up. pveceph purge does not even do it's job at cleaning up! Hillarious really!
Unable to create a new ceph cluster.... reinstall again again again...

And the documentation on how to fix this is ofc also lacking.

 

[AngryAdm]

WHYYYYYYYYYYYYYYYYYYY...... pveceph purge should have taken care of this!!!!!!

rm -rf /var/lib/ceph/
rm: cannot remove '/var/lib/ceph/osd/ceph-13': Device or resource busy
rm: cannot remove '/var/lib/ceph/osd/ceph-9': Device or resource busy
rm: cannot remove '/var/lib/ceph/osd/ceph-18': Device or resource busy
rm: cannot remove '/var/lib/ceph/osd/ceph-8': Device or resource busy
root@pve05:~#

 

[AngryAdm]

Now I have to go through this https://forum.proxmox.com/threads/sda-has-a-holder.97771/#post-423005
For every single OSD!

ANOYED!

 

[AngryAdm]

After fighting with PURGING ceph.... eventually I could install it and configure it on a node... and surprise surprise... the proxmox way!

I've had it with this bs.

 

[AngryAdm]

root@pve03:/var/lib# pveceph purge
^[[A
^[[Aunable to get monitor info from DNS SRV with service name: ceph-mon
2021-10-21T22:58:30.542+0200 7f5b48667280 -1 failed for service _ceph-mon._tcp
2021-10-21T22:58:30.542+0200 7f5b48667280 -1 monclient: get_monmap_and_config cannot identify monitors to contact
Error gathering ceph info, already purged? Message: rados_connect failed - No such file or directory
Error gathering ceph info, already purged? Message: rados_connect failed - No such file or directory
Removing config & keyring files
root@pve03:/var/lib# pveceph purge

^[[A
unable to get monitor info from DNS SRV with service name: ceph-mon
2021-10-21T22:58:31.258+0200 7fad876e8280 -1 failed for service _ceph-mon._tcp
2021-10-21T22:58:31.258+0200 7fad876e8280 -1 monclient: get_monmap_and_config cannot identify monitors to contact
Error gathering ceph info, already purged? Message: rados_connect failed - No such file or directory
Error gathering ceph info, already purged? Message: rados_connect failed - No such file or directory
Removing config & keyring files
root@pve03:/var/lib# pveceph purge
^[[A
unable to get monitor info from DNS SRV with service name: ceph-mon
2021-10-21T22:58:31.958+0200 7ff0643d4280 -1 failed for service _ceph-mon._tcp
2021-10-21T22:58:31.958+0200 7ff0643d4280 -1 monclient: get_monmap_and_config cannot identify monitors to contact
Error gathering ceph info, already purged? Message: rados_connect failed - No such file or directory
Error gathering ceph info, already purged? Message: rados_connect failed - No such file or directory
Removing config & keyring files
root@pve03:/var/lib# pveceph purge
unable to get monitor info from DNS SRV with service name: ceph-mon
2021-10-21T22:58:32.662+0200 7f9841481280 -1 failed for service _ceph-mon._tcp
2021-10-21T22:58:32.662+0200 7f9841481280 -1 monclient: get_monmap_and_config cannot identify monitors to contact
Error gathering ceph info, already purged? Message: rados_connect failed - No such file or directory
Error gathering ceph info, already purged? Message: rados_connect failed - No such file or directory
Removing config & keyring files
root@pve03:/var/lib# pveceph purge
unable to get monitor info from DNS SRV with service name: ceph-mon
2021-10-21T22:58:33.230+0200 7ff6f1eb4280 -1 failed for service _ceph-mon._tcp
2021-10-21T22:58:33.230+0200 7ff6f1eb4280 -1 monclient: get_monmap_and_config cannot identify monitors to contact
Error gathering ceph info, already purged? Message: rados_connect failed - No such file or directory
Error gathering ceph info, already purged? Message: rados_connect failed - No such file or directory
Removing config & keyring files
root@pve03:/var/lib# pveceph purge
unable to get monitor info from DNS SRV with service name: ceph-mon
2021-10-21T22:58:33.762+0200 7f7784742280 -1 failed for service _ceph-mon._tcp
2021-10-21T22:58:33.762+0200 7f7784742280 -1 monclient: get_monmap_and_config cannot identify monitors to contact
Error gathering ceph info, already purged? Message: rados_connect failed - No such file or directory
Error gathering ceph info, already purged? Message: rados_connect failed - No such file or directory
Removing config & keyring files
root@pve03:/var/lib# pveceph purge
'unable to get monitor info from DNS SRV with service name: ceph-mon
2021-10-21T22:58:38.434+0200 7f30007fc280 -1 failed for service _ceph-mon._tcp
2021-10-21T22:58:38.434+0200 7f30007fc280 -1 monclient: get_monmap_and_config cannot identify monitors to contact
Error gathering ceph info, already purged? Message: rados_connect failed - No such file or directory
Error gathering ceph info, already purged? Message: rados_connect failed - No such file or directory
Removing config & keyring files
root@pve03:/var/lib#


Should be it right? :/

NOOOOOOOOOOOOOOOOOOO

 

[camilocv]

A mí me pasa lo mismo y creo que no hay solución