Ceph | Trying to understand fault tolerance

A

Anotheruser

Member
Sep 21, 2022
70
22
13
I am currently looking into ceph and read several posts strongly discouraging a 2 Node setup.

Lets say you are running a 2 node, 2 replica, 1 minimum replica cluster with an additional monitor as third voting device.
From my understanding this would be a similar risk for data loss like running a raid1 / raidz1 pool (ignoring the fact that ceph has more potential failure points than a conventional disk + hba) or to put it different i can loose one disk / one node without loosing any data, if the second one also goes obviously the pool is lost.
Is this correct?

Does ceph store checksums or sth similar to zfs to ensure data integrity on every node or does it use / require a separate replica to verify / compare data integrity?

Thanks for any help :)
 
Anotheruser said:
From my understanding this would be a similar risk for data loss like running a raid1 / raidz1 pool
Click to expand...
yes and no. RAID is not host domain aware. RAID is meant to never have disks operate independently where ceph is. consequently, the rules by default would not allow operation with only one osd member present- and while you could override it, you should think long and hard about what it is you are expecting from your storage; most people expect to take data and retain it properly.

Anotheruser said:
i can loose one disk / one node without loosing any data, if the second one also goes obviously the pool is lost.
Click to expand...
This is not guaranteed. data integrity is provided by comparing the different pieces of pg to all members in the pg group. if you dont have any members to compare, the storage does not guarantee that what its giving you is what was written.

Again, the question you should be asking is "what am I expecting from the storage?" yes, you can operate in the manner you describe- but only if you dont care about the data.
 
What will happens if both nodes lose connection between and both nodes continues to write locally. How you will merge differences?
 
You must log in or register to reply here.
Top Bottom