Replication 3/2 vs 2/2 and read only

[sahostking]

Hi guys

We wanted to move to 2/2 for a bit while we wait for our new SSDs to arrive as we have limited storage space now in one cluster. However when doing so and moving from 3/2 to 2/2 we notice that all our VMs pause or become "read only" when Ceph is rebalancing if a disk is taken out and a new one is put back in.

Is this normal behaviour?

 

[mira]

Yes, that is expected behavior.

When you have 2/2 there are only 2 copies of each object available. But as you require a minimum of 2 copies for data to be writable (min_size), some objects won't be writable as soon as you remove one OSD.
That's because you only have 1 copy available for the objects which had copies on that OSD.

 

[sahostking]

That helps. I really don't want to use 2/1 as this is a production cluster. So will rather wait for the disks to arrive.

Thanks

 

[tom]

really don't want to use 2/1 as this is a production cluster

PLEASE use ONLY 3/2 if you care about your data.

 

[sahostking]

yes that's why I said I will wait for the extra disks to arrive

Thanks