Add new SSD's in productive HDD ceph cluster (with wrong device class)

[felipe]

Hi,

we have a productive ceph cluster (se config in file). We changed allready the device classes to hdd for all disks (it looks okay i think)
then i tried to: ceph osd crush rule create-replicated highspeedpool default host ssd
but throws the error: Error EINVAL: device class ssd does not exist
So mayme i need to add the ssd osd's first?
The problem ist that when i add the osd via the gui it will be in the wrong class (hdd because the raid controller does not support jbod and so the ssd's wont be recognized)
So how can i create the osd and set the divice class with ceph osd crush set-device-class ssd osd.X befor the osd#s are used in the hdd pool directly after creation?

thanks
philipp

---------------

 

[Alwin]

You will need to move the current pool to your new device class first. Otherwise they will stick with the default rule.
https://pve.proxmox.com/pve-docs/chapter-pveceph.html#pve_ceph_device_classes

 

[felipe]

You will need to move the current pool to your new device class first. Otherwise they will stick with the default rule.
https://pve.proxmox.com/pve-docs/chapter-pveceph.html#pve_ceph_device_classes

i dont understand how can i move the pool?
the rulseset is allready changed to:

rule replicated_ruleset {
id 0
type replicated
min_size 1
max_size 10
step take default class hdd
step chooseleaf firstn 0 type host
step emit
}

so all pools use the ruleset "replicated_ruleset" which has class hdd.
but i am afraid that the ssds will be marked as hdd (becauseof single raid0 disks - no jbod possible)
amd also i am not able to create a new rulesez für ssd

 

[Alwin]

Create the first (ssd) OSD and then you should be able to create the rule and use another pool to run on the SSD OSDs.

 

[felipe]

Create the first (ssd) OSD and then you should be able to create the rule and use another pool to run on the SSD OSDs.

yes i was thinking about that because it failed. but my main concern is that when i do a
pveceph osd create ... the osd will be marked as a hdd (instead of ssd - because no jbod) and then will be started directly. or does this command not start the osd like in the gui? if i add the osd in the gui definetly the ssd will end up in the wrong pool before i can change the device class manually.

 

[Alwin]

Why should the OSD not be recognized as SSD? This works pretty good in Nautilus. Also even if it is recognized wrongly, as soon as you set the device class the data migration will stop. Not much should happen there.

Otherwise you will need to add the OSD by hand (ceph-volume). Setting the device class through pveceph will come somewhere around pve-manager 6.2-11.

 

[felipe]

>> Why should the OSD not be recognized as SSD?
Because its on RAID0 with 1 Disk (no jbod)

Setting "osd_class_update_on_start = false" did the trick for me. Afterwards you can set it manually.