[SOLVED] Ceph bluestore WAL and DB size – WAL on external SSD

[semira uthsala]

Hi All,

I’m setting up ceph cluster with 3x node pve 6.2. each node got following disks

• 7x 6TB 7200 Enterprise SAS HDD
• 2x 3TB Enterprise SAS SSD
• 2x 400GB Enterprise SATA SSD

This setup previously used for old ceph (with filestore) cluster where it configured to use 2x 400GB SATA SSD to have journals for HDD OSD, and 2x 3TB SAS SSDs to have journals in the same disk.

Now I’m zapping all the drives and about the create bluestore on each.

My questions are

1. Is it possible to use 2x 400GB SATA SSD for rockDB for HDDs
( As I read rockDB partition capacity should be not less than 4% of block device capacity. And with the 7x 6TB HDDs I don’t have enough capacity for rockDB partitions on 2x 400GB SATA SSDs)

2. Can I set up WAL only on 2x 400GB SATA SSD partitions and use the same 7x 6TB HDDs block devices for rockDB? will this improve performance?

3. If not, how can I use this extra 2x 400GB SATA SSD on a useful way

 

[Alwin]

1. Is it possible to use 2x 400GB SATA SSD for rockDB for HDDs
( As I read rockDB partition capacity should be not less than 4% of block device capacity. And with the 7x 6TB HDDs I don’t have enough capacity for rockDB partitions on 2x 400GB SATA SSDs)

4% is a guestimate by the Ceph people. The RocksDB goes more in the direction of 3/30/300 GB for its DB tables. Should the DB not fit into the given space on the separate DB device, it will just overflow onto the data device. This will drop the performance but operation continues.

2. Can I set up WAL only on 2x 400GB SATA SSD partitions and use the same 7x 6TB HDDs block devices for rockDB? will this improve performance?

Yes you could, but they use 512MB by default and best work if the DB is on a faster device as well.

3. If not, how can I use this extra 2x 400GB SATA SSD on a useful way

Maybe in a separate pool.
https://pve.proxmox.com/pve-docs/chapter-pveceph.html#pve_ceph_device_classes

 

[semira uthsala]

Thanks for the reply Alwin

I already setup DB/WAL on the same device since Bluestore anyway faster than filestore.

Maybe in a separate pool.

Thanks, I will consider this.