NVMEs get detected as SSD-drives - ceph crush rule problem

[sigmarb]

Hi,

we just put our first NVMEs in the existing proxmox 6 & ceph nodes. Unfortunately the NVMEs are detected as type SSD.
As we want to have a new pool with only the faster NVMEs, we can not distinguish the disks by type:
(ceph osd crush rule create-replicated ....)

What can we do?

thank you.

 

[t.lamprecht]

Hi,

IIRC, currently NVMEs are always shown as SSDs in the Disk panel, they are a more specific type of SSD so it's not directly wrong.

But, I recently thought myself that it would be nice to use different types for them, we may do this with a future update.

As we want to have a new pool with only the faster NVMEs, we can not distinguish the disks by type:
(ceph osd crush rule create-replicated ....)

Note that the Disk panels type detection is not the same as the Ceph one.
Just use the hint that NVMe disks have "nvme" in their device path when selecting them.

 

[sigmarb]

Hi,

i dont quite get it. You do not just "add" disks to a pool, you create a crush map rule by

ceph osd crush rule create-replicated replicated_rule_ssd default host ssd

So it only gets replicated to this type of device. I'm not aware of another way to -as you recommend - "Just use the hint that NVMe disks have "nvme" in their device path when selecting them.".

Can you please elaborate?

thank you.

 

[sigmarb]

Just checked the manual. With luminous you can set device classes per device - its kind of overwriting the detected class:

https://ceph.io/community/new-luminous-crush-device-classes/

that should do the trick, will test.

 

[sigmarb]

works.

 

[Alwin]

So it only gets replicated to this type of device. I'm not aware of another way to -as you recommend - "Just use the hint that NVMe disks have "nvme" in their device path when selecting them.".

Thomas is talking about the GUI panel in your screenshot. That information is provided by proxmox. While Ceph might have its own classification of disks.

As you found out, in Ceph the device class can be set manually as well.