Using partition for OSD and WAL/DB

[igloo]

Hello!

Trying to set up brand new proxmox/ceph cluster. Got few problems:

1. Would it make sense to use SSD for wal/db? All osds are using HDDs, so I believe I'd benefit from using SSD for that.

2. Is it possible to use a partition instead of the whole drive to use for OSD while using a separate partition for SSD for wal/db? E.g. /dev/sda2 is a wal/db for /dev/sdb2, and so on..

Thank you!

 

[wolfgang]

Hi,

1. Would it make sense to use SSD for wal/db? All osds are using HDDs, so I believe I'd benefit from using SSD for that.

Yes if the SSD is fast, no if it is a consumer/prosumer SSD.

2. Is it possible to use a partition instead of the whole drive to use for OSD while using a separate partition for SSD for wal/db? E.g. /dev/sda2 is a wal/db for /dev/sdb2, and so on..

Yes, you can do that, normal enterprise SSD like the S3600 can only handle 4 osd's.
If the SSD dies all osd is offline and cannot recover. So you have to rebuild from the other 2 nodes

 

[shafeeks]

Hi,

While searching the forum for the same purpose, I found this post.

I am currently installing a cluster of 3 servers with Proxmox 6 + Ceph. I have 3 OSD 940GB SSD and 3 OSD SAS 4TB 7.2K on each server and Proxmox 6 is installed on 2x240G SSD RAIDZ1. I have used only 125G for Proxmox and have 115G Free?

For my 3 OSD SAS, I would like to use a partition of 115G on the SSD (disks where proxmox is installed) as Bluestore DB.
I created the partition (/dev/sda3) on each SSD 240G disk. Could you please telle me which partition type I need for this?

Is it possible to use a SSD partition for OSDs and how?

Thanks
Shafeek

 

[shafeeks]

hello
Is it possible on proxmox 6 with Ceph to push WAL/DB for slow OSDs on several SSD partitions and not the whole SSD disk? If yes, how to proceed?

Thanks for your update.

 

[Sarel Pretorius]

Hi,

Yes if the SSD is fast, no if it is a consumer/prosumer SSD.


Yes, you can do that, normal enterprise SSD like the S3600 can only handle 4 osd's.
If the SSD dies all osd is offline and cannot recover. So you have to rebuild from the other 2 nodes

Hi Wolgang, how would one create wal/dv using only partition of SSD? I have tried the following command but get error:

root@pve-a:~# pveceph osd create /dev/sdb -db_dev /dev/sda4 -wal_dev /dev/sda5
unable to get device info for '/dev/sda4' for type db_dev

/dev/sda is a single Enterprise SSD

 

[codingspiderfox]

Hi,

Yes if the SSD is fast, no if it is a consumer/prosumer SSD.


Yes, you can do that, normal enterprise SSD like the S3600 can only handle 4 osd's.
If the SSD dies all osd is offline and cannot recover. So you have to rebuild from the other 2 nodes

Even consumer SSDs are fast, so why would a consumer SSD not be beneficial over a HDD?

 

[tom]

Even consumer SSDs are fast

No, not for this workload.

You will find countless benchmarks about these topics, e.g. this blob post:
https://www.sebastien-han.fr/blog/2...-if-your-ssd-is-suitable-as-a-journal-device/

 

[codingspiderfox]

No, not for this workload.

You will find countless benchmarks about these topics, e.g. this blob post:
https://www.sebastien-han.fr/blog/2...-if-your-ssd-is-suitable-as-a-journal-device/

Shocking

 

[hmichl]

Experienced this issue too - in 7.3 still.

The reason is the kernel is not updating the partition table during OSD creation, if the disk is actively used/busy.
I put up a fix on github, hope it'll get merged soon:
https://github.com/proxmox/pve-storage/pull/12