Ceph pool - Wrong available disk

[alterman1994]

Hey Guys,

We had 24 OSDs (2 TB) drives with overall available space of 19.97 TB (replica size - 2 ) on our primary ceph-storage pool.

We added 10 more OSDs (same 2 TB drives) and having 25.42 TB only (while I'm expect to have ~ 28 TB approximately)

Funny thing, that when I added latest OSD (I added all of them 1 by 1), I got less disk available than I've had:



Could someone please explain why so? I somewhere lost mostly 3 TB of space. My pools configured:



Proxmox version: 6.4-13
Ceph version: 15.2.14

Thanks!

 

[gurubert]

The data available in a pool depends on the placement rules you have in the crush map. This may not always be 50% of your total capacity.

BTW: To not loose data I would recommend a pool size of at least 3.

 

[alterman1994]

@gurubert thanks, I will take a look. unfortunately, we cannot afford 3 replicas, too expensive setup.