Ceph removed per instructions - one node has errors


Renowned Member
Jul 8, 2014

I experimented with a four node ceph cluster and eventually decided it was not a good fit for me. I followed the instructions on the Proxmox support site and removed it from all the nodes. I have one node that continues to have errors posted to the logs. It started with lvm errors, which I researched and removed via the systemd disable command, but I am still left with crash logs every few seconds.

ceph crash ls:

Error initializing cluster client: ObjectNotFound('RADOS object not found (error calling conf_read_file)')

GUI error message:

Nov 21 09:30:07 darkbramble ceph-crash[832]: ERROR:ceph-crash:directory /var/lib/ceph/crash/posted does not exist; please create

I checked apt-cache and there are quite a few ceph packages installed. When I tested an apt remove ceph-common command, I didn't like some of the packages it wanted to remove (pve-manager, pve-ha-manager, pve-qemu-kvm...)

The ceph repository is disabled in the GUI.

Thanks for any help you can provide.
Thanks for your willingness to help!

This was a four node hyperconverged proxmox homelab. Using identical M900 Lenovo tiny units with 1TB SSD's on each. Ceph worked quite well and was able to do everything I wanted it to do. After a month or so, I found that I could do what I was really interested in doing with ZFS and HA. I can stand a few minutes of downtime if the node goes down while it brings up the VM on another node. So, I decided to remove the Ceph storage and followed the instructions here provided by a staff member:

Go to a nodes ceph tab and destroy first all OSDs, then all Managers and Monitors but not the last one.
On the last one just stop those two, then go to the command line and execute
pveceph purge
to remove the ceph packages for each node.

After the ceph storage was removed, I downsized the proxmox cluster to three nodes. Upon reviewing the logs, I'm seeing the above error messages on just one node. This wasn't even the last one I ran the purge command from. Not sure what happened to it.

The instructions you provided in the link above seems to have worked. No further messages in the syslog. I'll continue to monitor for the next few days. Thank you!



The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!