OSD Full and CEPH crashed completely - please help

berkaybulut

Member
Feb 8, 2023
19
0
6
I just received a few warnings that CEPH's OSD disks were full. When I started the process to install a new OSD, I started getting a got timeout (500) error. When I check the logs, I get the following errors. Currently Ceph is completely closed and I cannot open it.

Code:
root@*:~# journalctl -b -u "ceph-mon@*.service"
Feb 02 02:31:46 cmt6770 systemd[1]: Started ceph-mon@*.service - Ceph cluster monitor daemon.
Feb 02 02:31:46 cmt6770 systemd[1]: Started ceph-mon@*.service - Ceph cluster monitor daemon.
Feb 02 02:31:46 cmt6770 systemd[1]: ceph-mon@*.service: Main process exited, code=exited, status=28/n/a
Feb 02 02:31:46 cmt6770 systemd[1]: ceph-mon@*.service: Failed with result 'exit-code'.
Feb 02 02:31:46 cmt6770 systemd[1]: ceph-mon@*.service: Main process exited, code=exited, status=1/FAILURE
Feb 02 02:31:46 cmt6770 systemd[1]: ceph-mon@*.service: Failed with result 'exit-code'.
Feb 02 02:31:56 cmt6770 systemd[1]: ceph-mon@*.service: Scheduled restart job, restart counter is at 1.
Feb 02 02:31:56 cmt6770 systemd[1]: ceph-mon@*.service: Scheduled restart job, restart counter is at 1.
Feb 02 02:31:56 cmt6770 systemd[1]: Stopped ceph-mon@*.service - Ceph cluster monitor daemon.
Feb 02 02:31:56 cmt6770 systemd[1]: Started ceph-mon@*.service - Ceph cluster monitor daemon.
Feb 02 02:31:56 cmt6770 systemd[1]: Stopped ceph-mon@*.service - Ceph cluster monitor daemon.
Feb 02 02:31:56 cmt6770 systemd[1]: Started ceph-mon@*.service - Ceph cluster monitor daemon.
Feb 02 02:31:56 cmt6770 systemd[1]: ceph-mon@*.service: Main process exited, code=exited, status=28/n/a
Feb 02 02:31:56 cmt6770 systemd[1]: ceph-mon@*.service: Failed with result 'exit-code'.
Feb 02 02:31:56 cmt6770 systemd[1]: ceph-mon@*.service: Main process exited, code=exited, status=1/FAILURE
Feb 02 02:31:56 cmt6770 systemd[1]: ceph-mon@*.service: Failed with result 'exit-code'.
Feb 02 02:32:06 cmt6770 systemd[1]: ceph-mon@*.service: Scheduled restart job, restart counter is at 2.
Feb 02 02:32:06 cmt6770 systemd[1]: ceph-mon@*.service: Scheduled restart job, restart counter is at 2.
Feb 02 02:32:06 cmt6770 systemd[1]: Stopped ceph-mon@*.service - Ceph cluster monitor daemon.
Feb 02 02:32:06 cmt6770 systemd[1]: Started ceph-mon@*.service - Ceph cluster monitor daemon.
Feb 02 02:32:06 cmt6770 systemd[1]: Stopped ceph-mon@*.service - Ceph cluster monitor daemon.

Code:
root@*:~# ceph -s
got timeout