Is the Ceph replicate or duplicate files on 3 nodes

[UK SPEED]

Hello
How Data Is Stored In CEPH Cluster, I need to ask exactly to how the data have been read and written in the shared storage, Will the data replicate ( replication tasks time ) or it will be written and read at the same time on the ( shared storage ) without losing any chance to miss any data (duplicate date on the 3 nodes ).

I have a customer who need not lose any single request for any data.

What is the best solution for that case, please?

 

[shanreich]

When you write data to Ceph, then the write will only get acknowledged as soon as all 3 copies of the block have been written, there is no delay in replication as with e.g. ZFS.

 

[UK SPEED]

So, is data replicate or duplicate

 

[shanreich]

Not sure what you exactly mean with the difference - but the exact word doesn't really matter. If the write has been acknowledged then the written data has been written to all 3 OSDs - so as soon as you finished writing you got 3 copies.

 

[UK SPEED]

What about if the links between them are less then 100Mb/s, I mean in 3 different locations, what will happen?

 

[Falk R.]

Then you have a speed that feels like a floppy disk drive.
For Ceph, 10 GBit is the absolute minimum, I only build new clusters below 100 GBit in exceptional cases, but then with at least 25 GBit.
You may first need to read up on what Ceph is and how it works.

With 100MBit you could try ZFS replication. Depending on the change rate, this could work well.
You can also run the replication at intervals of only 1 minute, but it is always asynchronous.

There is no professional synchronous storage solution for 100 MBit.

 

[ness1602]

You can work for simple pve/ceph cluster with 2.5gbit networking, i made a few, but yes, 10g is the minimum if you are working in a >5 node cluster.

 

[Falk R.]

You can work for simple pve/ceph cluster with 2.5gbit networking, i made a few, but yes, 10g is the minimum if you are working in a >5 node cluster.

This is for Home Setups OK, but for Business is this Performance to poor.
We are all agree, 100MBit is not realistic usable.

 

[ness1602]

100mbit is even unrealistic for pve-zsync.

 

[UK SPEED]

Thank you, guys, for your reply, it is beneficial information, So 3 nodes on the same switch with 10GB ports /10GB speed. is that ok for a good solution?