Add new SSD's in productive HDD ceph cluster (with wrong device class)

felipe

Active Member
Oct 28, 2013
221
6
38
Hi,

we have a productive ceph cluster (se config in file). We changed allready the device classes to hdd for all disks (it looks okay i think)
then i tried to: ceph osd crush rule create-replicated highspeedpool default host ssd
but throws the error: Error EINVAL: device class ssd does not exist
So mayme i need to add the ssd osd's first?
The problem ist that when i add the osd via the gui it will be in the wrong class (hdd because the raid controller does not support jbod and so the ssd's wont be recognized)
So how can i create the osd and set the divice class with ceph osd crush set-device-class ssd osd.X befor the osd#s are used in the hdd pool directly after creation?

thanks
philipp

---------------
 

Attachments

  • crush.txt
    4.6 KB · Views: 4

felipe

Active Member
Oct 28, 2013
221
6
38
You will need to move the current pool to your new device class first. Otherwise they will stick with the default rule.
https://pve.proxmox.com/pve-docs/chapter-pveceph.html#pve_ceph_device_classes

i dont understand how can i move the pool?
the rulseset is allready changed to:

rule replicated_ruleset {
id 0
type replicated
min_size 1
max_size 10
step take default class hdd
step chooseleaf firstn 0 type host
step emit
}

so all pools use the ruleset "replicated_ruleset" which has class hdd.
but i am afraid that the ssds will be marked as hdd (becauseof single raid0 disks - no jbod possible)
amd also i am not able to create a new rulesez für ssd
 
Last edited:

Alwin

Proxmox Retired Staff
Retired Staff
Aug 1, 2017
4,617
453
88
Create the first (ssd) OSD and then you should be able to create the rule and use another pool to run on the SSD OSDs.
 

felipe

Active Member
Oct 28, 2013
221
6
38
Create the first (ssd) OSD and then you should be able to create the rule and use another pool to run on the SSD OSDs.
yes i was thinking about that because it failed. but my main concern is that when i do a
pveceph osd create ... the osd will be marked as a hdd (instead of ssd - because no jbod) and then will be started directly. or does this command not start the osd like in the gui? if i add the osd in the gui definetly the ssd will end up in the wrong pool before i can change the device class manually.
 

Alwin

Proxmox Retired Staff
Retired Staff
Aug 1, 2017
4,617
453
88
Why should the OSD not be recognized as SSD? This works pretty good in Nautilus. Also even if it is recognized wrongly, as soon as you set the device class the data migration will stop. Not much should happen there.

Otherwise you will need to add the OSD by hand (ceph-volume). Setting the device class through pveceph will come somewhere around pve-manager 6.2-11.
 

felipe

Active Member
Oct 28, 2013
221
6
38
>> Why should the OSD not be recognized as SSD?
Because its on RAID0 with 1 Disk (no jbod)

Setting "osd_class_update_on_start = false" did the trick for me. Afterwards you can set it manually.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!