Cannot add DB to OSD in Proxmox 6

[Ruffy91]

When trying to add a DB to an OSD when creating the OSD via Webinterface I get the following message:
"/dev/nvme0n1' is smaller than requested size '300059298201' bytes"

I tried setting the db size to auto, 1 GB, 3GB or 10 GB, the number never changes.

The OSD would consist of a 2TB HDD and part of a 32GB Optane module as DB.

 

[Ruffy91]

I-m very disappointed.
The new functionality is just not implemented yet. According to documentation and autocompletion in the CLI you can set the size but I can't find where it would be used in the code.
It says you can define it in GByte, else it will take either 10% of the OSD size or bluestore_block_db_size if defined in ceph.conf

It just takes bluestore_block_db_size which is 300GB if undefined

So I just set bluestore_block_db_size to 3GB in ceph.conf and enter whatever I want in the size field in the GUI because it is ignored.
After this creation worked.

 

[dcsapak]

2 things

1. the explicit size in the webgui should work but this is a bug (i opened one here: https://bugzilla.proxmox.com/show_bug.cgi?id=2292 )
2.

So I just set bluestore_block_db_size to 3GB in ceph.conf

you have to set it in bytes not in GB so put

bluestore_block_db_size = 3221225472

in

 

[Ruffy91]

Hi Dominik.

Thank you for opening the Bug report.

And yes this is what I did, enter it in Bytes, sorry for not being clear.

 

[Leah]

I have the same issue. Just wanted to create an OSD from a 6TB HDD with the DB/WAL on a 250GB SSD with the size set to auto. Didn't work. It falls back to the 10%/1% rule from the documentation. This isn't what I would expect from an "auto" mode.

 

[Leah]

Fixed it for now by defining the size manually. Is the need for this manual option something that changed with the upgrade to ceph nautilus?

 

[Alwin]

This isn't what I would expect from an "auto" mode.

See the bug tracker.

Is the need for this manual option something that changed with the upgrade to ceph nautilus?

No, that has been around since at least Ceph luminous.

 

[Leah]

No, that has been around since at least Ceph luminous.

Interesting, I'm 100% sure that I didn't need to define a size value with the 5.4 release and luminous.

 

[Leah]

See the bug tracker.

The issue in the bugtracker says that it ignores the given size, this wasn't the case for me. Defining a value smaller than my disk size worked. My problem was that the behavior of auto mode and the failing of it isn't really understandable

 

[Alwin]

Interesting, I'm 100% sure that I didn't need to define a size value with the 5.4 release and luminous.

In Ceph luminous the OSD creating tool changed, from ceph-disk to ceph-volume. With Ceph disk you also had to set the db/wal size in the config or create partitions first. There it wasn't possible to pass a custom size to ceph-disk.

The issue in the bugtracker says that it ignores the given size, this wasn't the case for me. Defining a value smaller than my disk size worked. My problem was that the behavior of auto mode and the failing of it isn't really understandable

ceph-volume seems to ignore the block-db-size option, that would create a custom sized DB partition or LV for the OSD. And then the default logic comes in.