[pve6] ceph lumi to nautilus guide problem: Required devices (block and data) not present for bluest

grin

grin

Renowned Member
Proxmox Subscriber
Dec 8, 2008
172
21
83
Hungary
grin.hu
# ceph-volume simple scan
stderr: lsblk: /var/lib/ceph/osd/ceph-2: not a block device
stderr: Bad argument "/var/lib/ceph/osd/ceph-2", expected an absolute path in /dev/ or /sys or a unit name: Invalid argument
Running command: /sbin/cryptsetup status /dev/sdd1
--> OSD 2 got scanned and metadata persisted to file: /etc/ceph/osd/2-d49cdeea-391e-41c1-ada8-fef3be0b6bc2.json
--> To take over management of this scanned OSD, and disable ceph-disk and udev, run:
--> ceph-volume simple activate 2 d49cdeea-391e-41c1-ada8-fef3be0b6bc2

etc...
and no surprise that activation isn't happy either:
# ceph-volume simple activate --all
--> activating OSD specified in /etc/ceph/osd/4-a95f413e-8f07-41ba-96e1-3a9ba73b8beb.json
--> Required devices (block and data) not present for bluestore
--> bluestore devices found: [u'data']
--> RuntimeError: Unable to activate bluestore OSD due to missing devices
....etc...
 
grin said:
/etc/ceph/osd/2-d49cdeea-391e-41c1-ada8-fef3be0b6bc2.json
Click to expand...
What is the content of the json file?

grin said:
stderr: Bad argument "/var/lib/ceph/osd/ceph-2", expected an absolute path in /dev/ or /sys or a unit name: Invalid argument
Click to expand...
Usually it finds the corresponding device, try the command with the specific device in question.
Code: 
ceph-volume simple scan /dev/sdX1
 
hi,

I have the exact same problem

# ceph-volume simple scan

Code: 
stderr: lsblk: /var/lib/ceph/osd/ceph-5: not a block device
 stderr: Bad argument "/var/lib/ceph/osd/ceph-5", expected an absolute path in /dev/ or /sys or a unit name: Invalid argument
Running command: /sbin/cryptsetup status /dev/sdh1
--> OSD 5 got scanned and metadata persisted to file: /etc/ceph/osd/5-acd2a716-4504-4163-b315-fee6b5a4ac42.json
--> To take over management of this scanned OSD, and disable ceph-disk and udev, run:
-->     ceph-volume simple activate 5 acd2a716-4504-4163-b315-fee6b5a4ac42
 stderr: lsblk: /var/lib/ceph/osd/ceph-36: not a block device
 stderr: Bad argument "/var/lib/ceph/osd/ceph-36", expected an absolute path in /dev/ or /sys or a unit name: Invalid argument
Running command: /sbin/cryptsetup status tmpfs
 stderr: blkid: error: tmpfs: No such file or directory
 stderr: lsblk: tmpfs: not a block device
--> Ignoring /var/lib/ceph/osd/ceph-36 because it's not a ceph-disk created osd.
 stderr: lsblk: /var/lib/ceph/osd/ceph-1: not a block device
 stderr: Bad argument "/var/lib/ceph/osd/ceph-1", expected an absolute path in /dev/ or /sys or a unit name: Invalid argument
Running command: /sbin/cryptsetup status /dev/sdc1
--> OSD 1 got scanned and metadata persisted to file: /etc/ceph/osd/1-ada95d3b-2271-417f-8957-092dbad83b62.json
--> To take over management of this scanned OSD, and disable ceph-disk and udev, run:
-->     ceph-volume simple activate 1 ada95d3b-2271-417f-8957-092dbad83b62
 stderr: lsblk: /var/lib/ceph/osd/ceph-2: not a block device
 stderr: Bad argument "/var/lib/ceph/osd/ceph-2", expected an absolute path in /dev/ or /sys or a unit name: Invalid argument
Running command: /sbin/cryptsetup status /dev/sde1
--> OSD 2 got scanned and metadata persisted to file: /etc/ceph/osd/2-f9a8609d-437b-4e11-93f7-6edc5e9fdbb4.json
--> To take over management of this scanned OSD, and disable ceph-disk and udev, run:
-->     ceph-volume simple activate 2 f9a8609d-437b-4e11-93f7-6edc5e9fdbb4
 stderr: lsblk: /var/lib/ceph/osd/ceph-0: not a block device
 stderr: Bad argument "/var/lib/ceph/osd/ceph-0", expected an absolute path in /dev/ or /sys or a unit name: Invalid argument
Running command: /sbin/cryptsetup status /dev/sdb1
--> OSD 0 got scanned and metadata persisted to file: /etc/ceph/osd/0-c905557d-3619-4d8b-9da5-cff66fa6340d.json
--> To take over management of this scanned OSD, and disable ceph-disk and udev, run:
-->     ceph-volume simple activate 0 c905557d-3619-4d8b-9da5-cff66fa6340d
 stderr: lsblk: /var/lib/ceph/osd/ceph-3: not a block device
 stderr: Bad argument "/var/lib/ceph/osd/ceph-3", expected an absolute path in /dev/ or /sys or a unit name: Invalid argument
Running command: /sbin/cryptsetup status /dev/sdf1
--> OSD 3 got scanned and metadata persisted to file: /etc/ceph/osd/3-0ed815ee-5cd5-4cfc-b2be-7e42b084ae6b.json
--> To take over management of this scanned OSD, and disable ceph-disk and udev, run:
-->     ceph-volume simple activate 3 0ed815ee-5cd5-4cfc-b2be-7e42b084ae6b
 stderr: lsblk: /var/lib/ceph/osd/ceph-4: not a block device
 stderr: Bad argument "/var/lib/ceph/osd/ceph-4", expected an absolute path in /dev/ or /sys or a unit name: Invalid argument
Running command: /sbin/cryptsetup status /dev/sdg1
--> OSD 4 got scanned and metadata persisted to file: /etc/ceph/osd/4-e5f9a7ec-0613-4a0d-b340-93219597cd46.json
--> To take over management of this scanned OSD, and disable ceph-disk and udev, run:
-->     ceph-volume simple activate 4 e5f9a7ec-0613-4a0d-b340-93219597cd46

Example from osd-0

Code: 
# cat /etc/ceph/osd/0-c905557d-3619-4d8b-9da5-cff66fa6340d.json
{
    "active": "ok",
    "block": {
        "path": "/dev/disk/by-partuuid/020eda5b-a959-4bfd-87e8-3dee5ad20564",
        "uuid": "020eda5b-a959-4bfd-87e8-3dee5ad20564"
    },
    "block_uuid": "020eda5b-a959-4bfd-87e8-3dee5ad20564",
    "bluefs": 1,
    "ceph_fsid": "e27fcfc8-4390-46e7-9f7e-4ada0c965785",
    "cluster_name": "ceph",
    "data": {
        "path": "/dev/sdb1",
        "uuid": "c905557d-3619-4d8b-9da5-cff66fa6340d"
    },
    "fsid": "c905557d-3619-4d8b-9da5-cff66fa6340d",
    "keyring": "AQCwo/BeVwY6FBAAuRBLU6UlMQoM3ztx6BVCRw==",
    "kv_backend": "rocksdb",
    "magic": "ceph osd volume v026",
    "mkfs_done": "yes",
    "ready": "ready",
    "require_osd_release": 12,
    "systemd": "",
    "type": "bluestore",
    "whoami": 0
}

If I try by hand:

Code: 
# ceph-volume simple scan /dev/sdb1

Running command: /sbin/cryptsetup status /dev/sdb1
-->  RuntimeError: --force was not used and OSD metadata file exists: /etc/ceph/osd/0-c905557d-3619-4d8b-9da5-cff66fa6340d.json


Code: 
# mount |grep osd
/
dev/sdc1 on /var/lib/ceph/osd/ceph-1 type xfs (rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota)
/dev/sdb1 on /var/lib/ceph/osd/ceph-0 type xfs (rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota)
/dev/sdf1 on /var/lib/ceph/osd/ceph-3 type xfs (rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota)
/dev/sde1 on /var/lib/ceph/osd/ceph-2 type xfs (rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota)
/dev/sdg1 on /var/lib/ceph/osd/ceph-4 type xfs (rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota)
/dev/sdh1 on /var/lib/ceph/osd/ceph-5 type xfs (rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota)
tmpfs on /var/lib/ceph/osd/ceph-36 type tmpfs (rw,relatime)
 
Last edited:
hi,

I tried it with one node and "ignored" the messages:

Code: 
# ceph-volume simple activate --all
--> activating OSD specified in /etc/ceph/osd/0-c905557d-3619-4d8b-9da5-cff66fa6340d.json
Running command: /bin/ln -snf /dev/sdb2 /var/lib/ceph/osd/ceph-0/block
Running command: /bin/chown -R ceph:ceph /dev/sdb2
Running command: /bin/systemctl enable ceph-volume@simple-0-c905557d-3619-4d8b-9da5-cff66fa6340d
 stderr: Created symlink /etc/systemd/system/multi-user.target.wants/ceph-volume@simple-0-c905557d-3619-4d8b-9da5-cff66fa6340d.service → /lib/systemd/system/ceph-volume@.service.
Running command: /bin/ln -sf /dev/null /etc/systemd/system/ceph-disk@.service
--> All ceph-disk systemd units have been disabled to prevent OSDs getting triggered by UDEV events
Running command: /bin/systemctl enable --runtime ceph-osd@0
Running command: /bin/systemctl start ceph-osd@0
--> Successfully activated OSD 0 with FSID c905557d-3619-4d8b-9da5-cff66fa6340d
--> activating OSD specified in /etc/ceph/osd/2-f9a8609d-437b-4e11-93f7-6edc5e9fdbb4.json
Running command: /bin/ln -snf /dev/sde2 /var/lib/ceph/osd/ceph-2/block
Running command: /bin/chown -R ceph:ceph /dev/sde2
Running command: /bin/systemctl enable ceph-volume@simple-2-f9a8609d-437b-4e11-93f7-6edc5e9fdbb4
 stderr: Created symlink /etc/systemd/system/multi-user.target.wants/ceph-volume@simple-2-f9a8609d-437b-4e11-93f7-6edc5e9fdbb4.service → /lib/systemd/system/ceph-volume@.service.
Running command: /bin/ln -sf /dev/null /etc/systemd/system/ceph-disk@.service
--> All ceph-disk systemd units have been disabled to prevent OSDs getting triggered by UDEV events
Running command: /bin/systemctl enable --runtime ceph-osd@2
Running command: /bin/systemctl start ceph-osd@2
--> Successfully activated OSD 2 with FSID f9a8609d-437b-4e11-93f7-6edc5e9fdbb4
--> activating OSD specified in /etc/ceph/osd/1-ada95d3b-2271-417f-8957-092dbad83b62.json
Running command: /bin/ln -snf /dev/sdc2 /var/lib/ceph/osd/ceph-1/block
Running command: /bin/chown -R ceph:ceph /dev/sdc2
Running command: /bin/systemctl enable ceph-volume@simple-1-ada95d3b-2271-417f-8957-092dbad83b62
 stderr: Created symlink /etc/systemd/system/multi-user.target.wants/ceph-volume@simple-1-ada95d3b-2271-417f-8957-092dbad83b62.service → /lib/systemd/system/ceph-volume@.service.
Running command: /bin/ln -sf /dev/null /etc/systemd/system/ceph-disk@.service
--> All ceph-disk systemd units have been disabled to prevent OSDs getting triggered by UDEV events
Running command: /bin/systemctl enable --runtime ceph-osd@1
Running command: /bin/systemctl start ceph-osd@1
--> Successfully activated OSD 1 with FSID ada95d3b-2271-417f-8957-092dbad83b62
--> activating OSD specified in /etc/ceph/osd/5-acd2a716-4504-4163-b315-fee6b5a4ac42.json
Running command: /bin/ln -snf /dev/sdh2 /var/lib/ceph/osd/ceph-5/block
Running command: /bin/chown -R ceph:ceph /dev/sdh2
Running command: /bin/systemctl enable ceph-volume@simple-5-acd2a716-4504-4163-b315-fee6b5a4ac42
 stderr: Created symlink /etc/systemd/system/multi-user.target.wants/ceph-volume@simple-5-acd2a716-4504-4163-b315-fee6b5a4ac42.service → /lib/systemd/system/ceph-volume@.service.
Running command: /bin/ln -sf /dev/null /etc/systemd/system/ceph-disk@.service
--> All ceph-disk systemd units have been disabled to prevent OSDs getting triggered by UDEV events
Running command: /bin/systemctl enable --runtime ceph-osd@5
Running command: /bin/systemctl start ceph-osd@5
--> Successfully activated OSD 5 with FSID acd2a716-4504-4163-b315-fee6b5a4ac42
--> activating OSD specified in /etc/ceph/osd/4-e5f9a7ec-0613-4a0d-b340-93219597cd46.json
Running command: /bin/ln -snf /dev/sdg2 /var/lib/ceph/osd/ceph-4/block
Running command: /bin/chown -R ceph:ceph /dev/sdg2
Running command: /bin/systemctl enable ceph-volume@simple-4-e5f9a7ec-0613-4a0d-b340-93219597cd46
 stderr: Created symlink /etc/systemd/system/multi-user.target.wants/ceph-volume@simple-4-e5f9a7ec-0613-4a0d-b340-93219597cd46.service → /lib/systemd/system/ceph-volume@.service.
Running command: /bin/ln -sf /dev/null /etc/systemd/system/ceph-disk@.service
--> All ceph-disk systemd units have been disabled to prevent OSDs getting triggered by UDEV events
Running command: /bin/systemctl enable --runtime ceph-osd@4
Running command: /bin/systemctl start ceph-osd@4
--> Successfully activated OSD 4 with FSID e5f9a7ec-0613-4a0d-b340-93219597cd46
--> activating OSD specified in /etc/ceph/osd/3-0ed815ee-5cd5-4cfc-b2be-7e42b084ae6b.json
Running command: /bin/ln -snf /dev/sdf2 /var/lib/ceph/osd/ceph-3/block
Running command: /bin/chown -R ceph:ceph /dev/sdf2
Running command: /bin/systemctl enable ceph-volume@simple-3-0ed815ee-5cd5-4cfc-b2be-7e42b084ae6b
 stderr: Created symlink /etc/systemd/system/multi-user.target.wants/ceph-volume@simple-3-0ed815ee-5cd5-4cfc-b2be-7e42b084ae6b.service → /lib/systemd/system/ceph-volume@.service.
Running command: /bin/ln -sf /dev/null /etc/systemd/system/ceph-disk@.service
--> All ceph-disk systemd units have been disabled to prevent OSDs getting triggered by UDEV events
Running command: /bin/systemctl enable --runtime ceph-osd@3
Running command: /bin/systemctl start ceph-osd@3
--> Successfully activated OSD 3 with FSID 0ed815ee-5cd5-4cfc-b2be-7e42b084ae6b

and rebooted the node ... it seems, it worked.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back