pveceph unable to get device info

[huky]

I have add two shannon pci-e ssd for ceph, but pveceph unable to get device info


# shannon-status -l
82:00:0: /dev/dfb Direct-IO G3S 1200GB 1200GB SS16704K7310005
81:00:0: /dev/dfa Direct-IO G3S 1200GB 1200GB SS16704K7310006

# pveceph createosd /dev/dfa
unable to get device info for 'dfa'

# ls -l /dev/df*
brw-rw---- 1 root disk 252, 0 Jun 26 16:27 /dev/dfa
brw-rw---- 1 root disk 252, 64 Apr 17 21:46 /dev/dfb

now, how could I to use it

 

[wolfgang]

Hi,

the device path is the problem.
/dev/dfa is a nonstandard device name.
Proxmox VE toolset has a whitelist of the following dev names.

    # whitelisting following devices
    # hdX: ide block device
    # sdX: sd block device
    # vdX: virtual block device
    # xvdX: xen virtual block device
    # nvmeXnY: nvme devices
    # cciss!cXnY: cciss devices

 

[huky]

Hi,

the device path is the problem.
/dev/dfa is a nonstandard device name.
Proxmox VE toolset has a whitelist of the following dev names.

    # whitelisting following devices
    # hdX: ide block device
    # sdX: sd block device
    # vdX: virtual block device
    # xvdX: xen virtual block device
    # nvmeXnY: nvme devices
    # cciss!cXnY: cciss devices

thansk, where is the list , can i add dfX into it?

 

[wolfgang]

This is in or code that means it is hardcoded and not a config file.

 

[Dark26]

pveceph createosd /dev/mmcblk0
unable to get device info for 'mmcblk0'

I believe it's the same problem with emmc storage?. No way to test this in my lab ?

 

[Alwin]

I believe it's the same problem with emmc storage?. No way to test this in my lab ?

Hm... you don't love your hardware? Even if it would work, the hardware will probably die very soon (besides a horrible performance).

 

[Dark26]

Hm... you don't love your hardware? Even if it would work, the hardware will probably die very soon (besides a horrible performance).

I love my hardware but i prefer to shrink my electricity bill at home, using low power baytrail mini pc for testing

 

[Alwin]

I love my hardware but i prefer to shrink my electricity bill at home, using low power baytrail mini pc for testing

Well, not that I warned you. You can always use the 'ceph-disk' utility directly to add OSDs.

 

[Dark26]

Well, not that I warned you. You can always use the 'ceph-disk' utility directly to add OSDs.

Thanks . I take a look.

 

[Dark26]

For the info :

ceph-disk prepare --cluster ceph --cluster-uuid e6a0e1ec-70c7-4a24-aaaa-9127fb59f5f9 --fs-type xfs /dev/mmcblk0

ceph-disk activate /dev/mmcblk0p1

I test it right now and it perform not so bad when i compare with my ganesha + glusterfs install for containers.

 

[Ingo S]

So, well, I have to dig up this thread because I stumbled upon the "device path" thing...

We have a 6 Node Cluster, of which are 4 Ceph Nodes, 8HDDs each Node and one Enterprise NVME SSD per Node. When I setup the ceph storage, I created a partition on the ssd for every osd, to serve as WAL device. Since the partitions have a naming scheme of /dev/nvmeXnYpZ the device path kicks in and I am unable to create OSDs with the NVME SSD as WAL Device.

How can I handle this?

greetings from the North Sea

 

[Alwin]

@Ingo S, as this is an old thread and refers to PVE 5 with ceph-disk (doesn't exist anymore), can you please open up a new thread? Thanks.