Ceph question : Will 2/2 size on a 4 host setup with 4 osd should work?

[DC-CA1]

in case 1 OSD / Node reboot , and we still have 3 node / OSD online

will the pool lock i/o if configured at 2/2 ?

im asking cause my collegue told me that during his test during a node reboot the VMs was not responding to i/o anymore with 2/2 but correctly with 2/1 on a 4 node lab

 

[gurubert]

Never run size=2 min_size=1 in production.
Always use size=3 min_size=2 for any valuable data.

When running size=2 min_size=2 and rebooting one node there are placement groups (and objects) with only one active copy in the cluster. PGs with less than min_size copies go into read-only mode. That translates into IO errors in the VM.

 

[DC-CA1]

Never run size=2 min_size=1 in production.
Always use size=3 min_size=2 for any valuable data.

When running size=2 min_size=2 and rebooting one node there are placement groups (and objects) with only one active copy in the cluster. PGs with less than min_size copies go into read-only mode. That translates into IO errors in the VM.

i am aware of that. its a lab .

but this do not answer the question , is 2/2 suppose to still serve on a 4 nodes / with 1 osd each ? i think answer is yes.
even with 3/2 is it suppose to keep serving right ?

maybe it was just a remote control issue .

 

[DC-CA1]

Never run size=2 min_size=1 in production.
Always use size=3 min_size=2 for any valuable data.

When running size=2 min_size=2 and rebooting one node there are placement groups (and objects) with only one active copy in the cluster. PGs with less than min_size copies go into read-only mode. That translates into IO errors in the VM.

PG are suppose to be spread on all 4 OSD in 2/2 already no , and still serve ?

 

[gurubert]

is 2/2 suppose to still serve on a 4 nodes / with 1 osd each ?

No, as some objects have not enough copies (less than min_size) and are in read-only.
This causes IO errors in VMs.

 

[gurubert]

Running with min_size=1 guarantees data loss.

 

[DC-CA1]

Running with min_size=1 guarantees data loss.

hi Gurubert its not the topic here but thx for the repeated warning lol

so 2/2 will put a 4 host 4 osd on read only per mecanism ?

 

[DC-CA1]

@gurubert is a 3/2 is suppose to keep serving in a 4 node / 4 osd scenarion if a node reboot ?

 

[DC-CA1]

@gurubert is a 3/2 is suppose to keep serving in a 4 node / 4 osd scenarion if a node reboot ?

@gurubert i think i was wrong about my understanding of CEPH ,
i tough a pool was replicating 100% of a time on all OSD by default.
this is incorrect right ?
if i have 2/2 , the data is on 3 OSD not 4 ? ( active splitted on 2 OSD and a 2 copy on 2 other OSD for a total of 3 ?