I think your Ceph cluster is working as intended. If you have the default replica = 3, when only two nodes are running, your cluster is in an unhealthy state until the third node comes back. This still allowes you to write data to the cluster, to the best of my knowledge, until only one server...