unable to get device info for '/dev/skd0'

[VonChair]

I have a three node cluster with an existing CEPH cluster running 14.2.9. Each host has 6 OSDs as part of the cluster and this works great. I have added 3x1TB PCIe EMLC SSDs to the cluster. One of these is in each node. My goal is to create a separate pool with only 3 OSD where I can store VM OS disk as my existing cluster is made of spinning disks so this will be better performance. When attempting to create an OSD however I get the error 'unable to get device name for'.

_james@existclust-01:~$ sudo pveceph osd create /dev/skd0
unable to get device info for '/dev/skd0'

Upon further investigation I suspect this is due to https://forum.proxmox.com/threads/pveceph-unable-to-get-device-info.44927/post-214510 this which tells me that I have a device name which is not compatible with pveceph.

Since I have an existing cluster already, I can not use

sudo ceph-volume lvm create --bluestore --data /dev/skd0

or I will be adding the OSD to the existing pool(I tried it and it does this so I removed it.) and since I wish for two separate pools, this is not an option for me from what I can tell.

Is there a way that an OSD can be created with a device name that is not standard? It is a real physical device so it's not like one of these threads trying to create OSDs on iSCSI LUNs or something crazy like this.

 

[fabian]

the easiest way is probably to just split those pools by device class. also, your view is backwards - it's not OSDs that are assigned to pools, but pools using specific OSDs (via the CRUSH map).

 

[VonChair]

Thanks for pointing me in the right direction. Looks like I've got a bit of work to do. I got the new pool online but since I don't have any rules for the other one, I've got to get to work learning a bit more about CEPH. This is resolved.