Howto add DB device to BlueFS

cmonty14

Well-Known Member
Mar 4, 2014
343
5
58
Hi,

I have created OSD on HDD w/o putting DB on faster drive.

In order to improve performance I have now a single SSD drive with 3.8TB.

Questions:
How can I add DB device for every single OSD to this new SSD drive?
Which parameter in ceph.conf defines the size for the DB?
Can you confirm that this parameter in ceph.conf will create 100GB for DB for each OSD?
[osd]
osd journal size = 102400

How can I increase the DB size later in case it's insufficient?

THX
 
Hi,

I have created OSD on HDD w/o putting DB on faster drive.

In order to improve performance I have now a single SSD drive with 3.8TB.

Questions:
How can I add DB device for every single OSD to this new SSD drive?

I have not validated the procedure, but you can attempt Migrating Bluestore block.db

Which parameter in ceph.conf defines the size for the DB?

bluestore_block_db_size

Can you confirm that this parameter in ceph.conf will create 100GB for DB for each OSD?
[osd]
osd journal size = 102400




I can confirm that WILL NOT do so -- that is for filestore not bluestore.

 
OK.
I modified /etc/ceph/ceph.conf by adding this in [global]:
bluestore_block_db_size = 53687091200
This should create RockDB with size 50GB.

Then I wanted to move DB to a new device (SSD) that is not formatted:
root@ld5505:~# ceph-bluestore-tool bluefs-bdev-new-db –-path /var/lib/ceph/osd/ceph-76 --dev-target /dev/sdbk
too many positional options have been specified on the command line


Checking the content of /var/lib/ceph/osd/ceph-76 it appears that there's no link to block.db:
root@ld5505:~# ls -l /var/lib/ceph/osd/ceph-76/
insgesamt 52
-rw-r--r-- 1 ceph ceph 418 Aug 27 11:08 activate.monmap
lrwxrwxrwx 1 ceph ceph 93 Aug 27 11:08 block -> /dev/ceph-8cd045dc-9eb2-47ad-9668-116cf425a66a/osd-block-9c51bde1-3c75-4767-8808-f7e7b58b8f97
-rw-r--r-- 1 ceph ceph 2 Aug 27 11:08 bluefs
-rw-r--r-- 1 ceph ceph 37 Aug 27 11:08 ceph_fsid
-rw-r--r-- 1 ceph ceph 37 Aug 27 11:08 fsid
-rw------- 1 ceph ceph 56 Aug 27 11:08 keyring
-rw-r--r-- 1 ceph ceph 8 Aug 27 11:08 kv_backend
-rw-r--r-- 1 ceph ceph 21 Aug 27 11:08 magic
-rw-r--r-- 1 ceph ceph 4 Aug 27 11:08 mkfs_done
-rw-r--r-- 1 ceph ceph 41 Aug 27 11:08 osd_key
-rw-r--r-- 1 ceph ceph 6 Aug 27 11:08 ready
-rw-r--r-- 1 ceph ceph 3 Aug 27 11:08 require_osd_release
-rw-r--r-- 1 ceph ceph 10 Aug 27 11:08 type
-rw-r--r-- 1 ceph ceph 3 Aug 27 11:08 whoami

root@ld5505:~# more /var/lib/ceph/osd/ceph-76/bluefs
1


How can I create DB on SSD device for existing OSDs?

THX
 
Some of the prospects, I have written on your other post [0].

To create a new DB on another device, use your command with an prefixed variable assignment [1]. Your command seems to only have the wrong dash assigned (–-path).

Besides the addition of the DB, you need to make sure that the LV tag for the ceph.db_device is updated, otherwise the OSD won't start.

As said in the other thread, re-creating the OSD might be easier and it will make sure that the on-disk format for the OSDs are current.

[0] https://forum.proxmox.com/threads/bluefs-spillover-detected-on-30-osd-s.56230/#post-264623
[1] https://www.spinics.net/lists/ceph-users/msg54305.html
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!