Ceph pool size 2/2 on 3-server Cluster

D

Deepen Dhulla

Renowned Member
Mar 1, 2016
84
17
73
48
Mumbai
deependhulla.com
Want to checkout what happen if Ceph pool size 2/2 instead of 2/3 on 3-server Proxmox-Ceph-Cluster , and if one Server goes down, would it be still able work in production like 2/3 setup...or will it halt all.

Reason is that we would be having PBS Setup to take daily backup. for Data Safety.
so today its 3 Server, when its 4 server, we would have more space, and better iops and bit low load on ceph network.

Want Proxmox team & community feedback on it. While creating a test setup did-not find any down-time effect on any VM & LXC. yet would like to know. so that if any thing is missed.

Thanks in advance., your feedback would help me if something is missed.
-Deepen.
 
2/2 means that there are 2 copies of each object, and you need at least 2 copies of each object for I/O to work. This means that as soon as you lose one OSD, I/O is completely blocked for some objects. This is also why I would strongly recommend against this setup, since as soon as something bad happens, I/O is (at least partly) blocked until recovery / rebalance has finished. If during this downtime something else happens (another OSD on another node goes down for instance), you might even lose some data completely.
 
  • Like
Reactions: tom
Ya I too had same thought about : as soon as you lose one OSD, I/O is completely blocked for some objects...But to my surprise it did not, atleast in my test senerios...where heavy data Read/write/update via custom script was done. thus thought to get community feedback on same.
 
It will work after recovering, when you have 3 nodes. So maybe that's why this worked in your case? Depending on how much data needs to be recovered, recovery can be quite fast. Newly written data will also be immediately available.
 
You must log in or register to reply here.
Top Bottom