[SOLVED] osd not being created

[mjoconr]

Hi
I've been having an issue with the creation of OSD's, most times I can add a disk and it apears on the OSD page and ceph osd tree show the newly adding drive.

But a number of times I've add the drive and I get something like the below which looks like its done but the drive is added to the OSD list.

At least once I've done the add and it was not there but a few mins later it apears.

Does anyone have any idea whats going wrong ?

Mike

Task viewer: Ceph OSD sdb - Create
OutputStatus
Stop
create OSD on /dev/sdb (bluestore)
Caution: invalid backup GPT header, but valid main header; regenerating
backup header from main header.

****************************************************************************
Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
verification and recovery are STRONGLY recommended.
****************************************************************************
GPT data structures destroyed! You may now partition the disk using fdisk or
other utilities.
Creating new GPT entries.
The operation has completed successfully.
Setting name!
partNum is 0
REALLY setting name!
The operation has completed successfully.
Setting name!
partNum is 1
REALLY setting name!
The operation has completed successfully.
The operation has completed successfully.
meta-data=/dev/sdb1 isize=2048 agcount=4, agsize=6400 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0
data = bsize=4096 blocks=25600, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
log =internal log bsize=4096 blocks=864, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
The operation has completed successfully.
TASK OK

 

[mjoconr]

I've tried everything I can think of to get the new OSD to be created and enable, but it does not appear even though I have zapped the drive rebooted then added it. Zeroed the drive for the first Gig of data.
I have no idea what my next steps should be to add this drive.

 

[Jarek]

Add osd manually:

ceph-disk prepare --bluestore /dev/sdX  --osd-id {id} --osd-uuid `uuidgen`
ceph-disk activate /dev/sdX1

 

[mjoconr]

Ok that worked, but why did not it not work using the pveceph createosd ?

root@pve:~# ceph-disk zap /dev/sdb
10+0 records in
10+0 records out
10485760 bytes (10 MB, 10 MiB) copied, 0.0983489 s, 107 MB/s
/dev/sdb1: 4 bytes were erased at offset 0x00000000 (xfs): 58 46 53 42
10+0 records in
10+0 records out
10485760 bytes (10 MB, 10 MiB) copied, 0.0802673 s, 131 MB/s
Caution: invalid backup GPT header, but valid main header; regenerating
backup header from main header.

Warning! Main and backup partition tables differ! Use the 'c' and 'e' options
on the recovery & transformation menu to examine the two tables.

Warning! One or more CRCs don't match. You should repair the disk!

****************************************************************************
Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
verification and recovery are STRONGLY recommended.
****************************************************************************
GPT data structures destroyed! You may now partition the disk using fdisk or
other utilities.
Creating new GPT entries.
The operation has completed successfully.
root@pve:~# ceph-disk prepare --bluestore /dev/sdb --osd-id 14 --osd-uuid `uuidgen`
Setting name!
partNum is 0
REALLY setting name!
The operation has completed successfully.
Setting name!
partNum is 1
REALLY setting name!
The operation has completed successfully.
The operation has completed successfully.
meta-data=/dev/sdb1 isize=2048 agcount=4, agsize=6400 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0
data = bsize=4096 blocks=25600, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
log =internal log bsize=4096 blocks=864, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
The operation has completed successfully.
root@pve:~# partprobe
root@pve:~# ceph-disk activate /dev/sdb1
Removed /run/systemd/system/ceph-osd.target.wants/ceph-osd@14.service.
Created symlink /run/systemd/system/ceph-osd.target.wants/ceph-osd@14.service → /lib/systemd/system/ceph-osd@.service.
root@pve:~#