[SOLVED] ceph is hung

huky

Active Member
Jul 1, 2016
61
1
28
40
Chongqing, China
i add a big disk as OSD to ceph cluster. and after one day,, my ceph is hung.

command 'ceph -s' hung

gui 'got timeout(500)'

how can i make it work?

in th process, i found disk full by /var/lib/ceph/mon/ceph-$NODE/store.db
now the node003 is 97%, node001 and node002 are full and i stop mon on the nodes.
 

Alwin

Proxmox Staff Member
Staff member
Aug 1, 2017
4,617
430
88
now the node003 is 97%, node001 and node002 are full and i stop mon on the nodes.
Ceph will not work without the MONs. You will need to provide more space to the partition where the store.db is located.
 

huky

Active Member
Jul 1, 2016
61
1
28
40
Chongqing, China
th
Ceph will not work without the MONs. You will need to provide more space to the partition where the store.db is located.
thanks, i created mon on other nodes with enough disk, after a few hours the size of mon (/var/lib/ceph/mon/ceph-$nodename/store.db/) is equal, but ceph is still hung.


I got it.
refer :
"REMOVING MONITORS FROM AN UNHEALTHY CLUSTER" in https://docs.ceph.com/en/latest/rados/operations/add-or-rm-mons/

delete two node and ceph cmd worked
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!