CEPH: Use lvm volume as block.db device ?

[Belokan]

Hello all,

Before being flamed please note that my question is related to a @home infrastructure and is not supposed to be deployed in any kind of production !

My Proxmox infrastructure is based on 3 nodes with an SSD used for OS and a single dedicated disk used as CEPH OSD (bluestore).
As I do not use the data lvm volume local storage (mounted under /var/lib/vz) I'd like to use it for block.db instead.
Is it possible to pass a volume to pveceph like --journal_dev=/dev/pve/blockdb ? I've planed to reduce /dev/pve/data volume & ext4 to about 5GB and then create a new /dev/pve/blockdb volume.

If this is possible, can I change the block.db location on "live OSD" or should I destroy my OSD and recreate them with the new journal_dev one by one ?

Thanks a lot in advance !

Olivier

 

[Alwin]

Is it possible to pass a volume to pveceph like --journal_dev=/dev/pve/blockdb ? I've planed to reduce /dev/pve/data volume & ext4 to about 5GB and then create a new /dev/pve/blockdb volume.

AFAIK, it is not tested but should work. While it may not degrade your performance or make your OS sluggish in home use, I DO NOT recommend it. From my experience, it will kill performance and makes the cluster unstable especially when under load.

If this is possible, can I change the block.db location on "live OSD" or should I destroy my OSD and recreate them with the new journal_dev one by one ?

You need to re-create the OSD.

 

[Belokan]

Thanks a lot for your answer Alwin ! Is your answer (first part regarding performance) valid if instead of an LVM volume I create a dedicated partition on the boot SSD ?

Have a nice day.

 

[Alwin]

In either case more IO is done on the SSD containing the OS. The writes of the DB/WAL (small writes) and the writes of the OS are competing.