Ceph loses total space, when close to full.

[potetpro]

Lost almost 100GB from startup, to 88% full.
We are currently running 3 nodes, with 2 500GB SSD's in each node, with replication 3, and minimum 2. (we are planning to add new nodes in a few months.)
Is this normal? of so why?

Thanks

 

[Alwin]

The disk with the highest fill level will dominate how much data can be written to the pool. As with the default replication factor, 3x copies are made. The pool will only show the data that can be written, before duplication.

 

[potetpro]

OK, makes sense. I see now that there is a small unbalance between the OSD's. Thanks Alwin