unable to get device info for '/dev/skd0'

VonChair

Member
Oct 23, 2020
6
1
6
I have a three node cluster with an existing CEPH cluster running 14.2.9. Each host has 6 OSDs as part of the cluster and this works great. I have added 3x1TB PCIe EMLC SSDs to the cluster. One of these is in each node. My goal is to create a separate pool with only 3 OSD where I can store VM OS disk as my existing cluster is made of spinning disks so this will be better performance. When attempting to create an OSD however I get the error 'unable to get device name for'.

Code:
_james@existclust-01:~$ sudo pveceph osd create /dev/skd0
unable to get device info for '/dev/skd0'

Upon further investigation I suspect this is due to https://forum.proxmox.com/threads/pveceph-unable-to-get-device-info.44927/post-214510 this which tells me that I have a device name which is not compatible with pveceph.

Since I have an existing cluster already, I can not use
Code:
sudo ceph-volume lvm create --bluestore --data /dev/skd0
or I will be adding the OSD to the existing pool(I tried it and it does this so I removed it.) and since I wish for two separate pools, this is not an option for me from what I can tell.

Is there a way that an OSD can be created with a device name that is not standard? It is a real physical device so it's not like one of these threads trying to create OSDs on iSCSI LUNs or something crazy like this.
 
the easiest way is probably to just split those pools by device class. also, your view is backwards - it's not OSDs that are assigned to pools, but pools using specific OSDs (via the CRUSH map).
 
Thanks for pointing me in the right direction. Looks like I've got a bit of work to do. I got the new pool online but since I don't have any rules for the other one, I've got to get to work learning a bit more about CEPH. This is resolved.