CEPH pool without replication?

[encore]

Hi,

is it possible to create a CEPH pool that does not replicate and where I have 100% of the storage available? We are talking about temporary data that is unimportant and may be lost.

Setting a pool with size/min to 1/1 does not mean that it does not replicate. Hence the question.

 

[dcsapak]

i mean it should be possible ( you might need to fallback to the ceph commandline instead of using pve cli/gui) but why would you might want that?

 

[encore]

but why would you might want that

because I have temporary data that is unimportant and I don't want to waste disk space.

I can't find a cli statement to accomplish this. I only find something about replicated_rule or erasure crush rule when creating the pool https://docs.ceph.com/en/latest/rados/operations/pools/ But that doesn't do me any good because erasure also contains replication. So my question if someone knows a way to create a pool completely without replication and wasting space.

 

[dcsapak]

yes you have to create a normal pool, but with '--size 1 --min_size 1'

 

[Klaus Steinberger]

because I have temporary data that is unimportant and I don't want to waste disk space.

I can't find a cli statement to accomplish this. I only find something about replicated_rule or erasure crush rule when creating the pool https://docs.ceph.com/en/latest/rados/operations/pools/ But that doesn't do me any good because erasure also contains replication. So my question if someone knows a way to create a pool completely without replication and wasting space.

i would strongly recommend to not even think about such a configuration, even not for temporary data. CEPH is definitly not designed for such a use case. It has also good reasons that you should not create a pool with size 2 min_size 1, use pool size 3 with min_size 2. Loss of data in such a crippled configuration is foreseeable.

For temporary data use other filesystems / storage techniques.

 

[encore]

i would strongly recommend to not even think about such a configuration, even not for temporary data. CEPH is definitly not designed for such a use case. It has also good reasons that you should not create a pool with size 2 min_size 1, use pool size 3 with min_size 2. Loss of data in such a crippled configuration is foreseeable.

For temporary data use other filesystems / storage techniques.

thank you for the helpful contribution

 

[benoitc]

i would strongly recommend to not even think about such a configuration, even not for temporary data. CEPH is definitly not designed for such a use case. It has also good reasons that you should not create a pool with size 2 min_size 1, use pool size 3 with min_size 2. Loss of data in such a crippled configuration is foreseeable.

For temporary data use other filesystems / storage techniques.

this imply to have also local disks for that. There should be a way to reuse the space handled by the OSD to handle suc things. I think one interresting feature would be able to launch a local vm associated to local a local OSD if possible. But last time I read it wasn't feasible with CEPH.

One question is does anyone know how cisco hyperflex works in such case ? At a first glance it is doing like CEPH:

When you migrate a virtual machine to a new location, the HX Data Platform does not require data to be moved. This approach significantly reduces the impact and cost of moving virtual machines among systems.

https://www.cisco.com/c/en/us/produ...perflex-hx-series/white-paper-c11-736814.html

with a replication of 3 and minimum 2 will it be able to continue to work during the replacement/upgrade of a node?

 

[Klaus Steinberger]

this imply to have also local disks for that. There should be a way to reuse the space handled by the OSD to handle suc things. I think one interresting feature would be able to launch a local vm associated to local a local OSD if possible. But last time I read it wasn't feasible with CEPH.

Again CEPH was never designed for this, also the additional software layer of CEPH bogs down performance, so maybe for some fast access of temporary data a local SSD is much better. But be aware that you loose HA in this case .....

One question is does anyone know how cisco hyperflex works in such case ? At a first glance it is doing like CEPH:



with a replication of 3 and minimum 2 will it be able to continue to work during the replacement/upgrade of a node?

yes with 3/2 the cluster continues to work during replacement/upgrade. You should set "noout" flag for OSDs during maintenance

 

[brosky]

bumping up this.

I'm in a situation that on a proxmox cluster with small local storage I need to map several 8TB block devices that will be used for a hadoop cluster.
Having in the same LAN a working Ceph, i'm considering creating a pool with min_size 1.
The data from that pool will be replicated by HDFS but ... just for the sake of conversation... what happens when:

- ok, one pool without replication. Blocks from that pool are distributed in PG's across the cluster. One OSD goes down. What happens ?

 

[itNGO]

- ok, one pool without replication. Blocks from that pool are distributed in PG's across the cluster. One OSD goes down. What happens ?

nothing left and you loose all.... with much luck when OSD comes up again you might have access again.... bad idea at all...