[SOLVED] Problem creating Ceph OSD

[frantek]

Hi,

I've setup a 3 node Cluster with Ceph for some tests. Today I tried to add new disk and thougt it may be a good idea to use the button "Initialize disk with GPT" on the first node. I was wrong. On the node where I did so I'm unable to add the disk to Ceph.

root@pve01:~# pveceph createosd /dev/sdg --journal_dev /dev/sdb
create OSD on /dev/sdg (xfs)
using device '/dev/sdb' for journal
Caution: invalid backup GPT header, but valid main header; regenerating
backup header from main header.

****************************************************************************
Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
verification and recovery are STRONGLY recommended.
****************************************************************************
GPT data structures destroyed! You may now partition the disk using fdisk or
other utilities.
Creating new GPT entries.
The operation has completed successfully.
WARNING:ceph-disk:OSD will not be hot-swappable if journal is not the same device as the osd data
Setting name!
partNum is 3
REALLY setting name!
Could not create partition 4 from 0 to 10485759
Unable to set partition 4's name to 'ceph journal'!
Could not change partition 4's type code to 45b0969e-9b03-4f30-b4c6-b4b80ceff106!
Error encountered; not saving changes.
ceph-disk: Error: Command '['/sbin/sgdisk', '--new=4:0:+5120M', '--change-name=4:ceph journal', '--partition-guid=4:2416d521-b875-4065-ac18-7fdde5efac0b', '--typecode=4:45b0969e-9b03-4f30-b4c6-b4b80ceff106', '--mbrtogpt', '--', '/dev/sdb']' returned non-zero exit status 4
command 'ceph-disk prepare --zap-disk --fs-type xfs --cluster ceph --cluster-uuid e9a07274-cba6-4c72-9788-a7b65c93e477 --journal-dev /dev/sdg /dev/sdb' failed: exit code 1

I've already tried to use "dd" and "gdisk" without success. How to fix this?

On the other nodes where I refrained from using "Initialize disk with GPT" the command worked with out any problems. BTW: I use the commandline command as the web interface does not allow me to select my journal SSD for some odd reason. It only presents my system SSD for journal usage.

TIA
Matthias

 

[wolfgang]

Hi,

do you have free space to create a 5GB part on /dev/sdb?

 

[udo]

Hi,

I've setup a 3 node Cluster with Ceph for some tests. Today I tried to add new disk and thougt it may be a good idea to use the button "Initialize disk with GPT" on the first node. I was wrong. On the node where I did so I'm unable to add the disk to Ceph.

root@pve01:~# pveceph createosd /dev/sdg --journal_dev /dev/sdb
create OSD on /dev/sdg (xfs)
using device '/dev/sdb' for journal
Caution: invalid backup GPT header, but valid main header; regenerating
backup header from main header.

****************************************************************************
Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
verification and recovery are STRONGLY recommended.
****************************************************************************
GPT data structures destroyed! You may now partition the disk using fdisk or
other utilities.
Creating new GPT entries.
The operation has completed successfully.
WARNING:ceph-disk:OSD will not be hot-swappable if journal is not the same device as the osd data
Setting name!
partNum is 3
REALLY setting name!
Could not create partition 4 from 0 to 10485759
Unable to set partition 4's name to 'ceph journal'!
Could not change partition 4's type code to 45b0969e-9b03-4f30-b4c6-b4b80ceff106!
Error encountered; not saving changes.
ceph-disk: Error: Command '['/sbin/sgdisk', '--new=4:0:+5120M', '--change-name=4:ceph journal', '--partition-guid=4:2416d521-b875-4065-ac18-7fdde5efac0b', '--typecode=4:45b0969e-9b03-4f30-b4c6-b4b80ceff106', '--mbrtogpt', '--', '/dev/sdb']' returned non-zero exit status 4
command 'ceph-disk prepare --zap-disk --fs-type xfs --cluster ceph --cluster-uuid e9a07274-cba6-4c72-9788-a7b65c93e477 --journal-dev /dev/sdg /dev/sdb' failed: exit code 1

I've already tried to use "dd" and "gdisk" without success. How to fix this?

On the other nodes where I refrained from using "Initialize disk with GPT" the command worked with out any problems. BTW: I use the commandline command as the web interface does not allow me to select my journal SSD for some odd reason. It only presents my system SSD for journal usage.

TIA
Matthias

Hi,
what kind of disk is this?

Have you tried to zap the disk before?

sgdisk -Z /dev/sdb

Udo

 

[frantek]

Thanks for the replies. The problem was that device names seem to be moving targets. I've created the first OSDs with sdb as log destination. Later I've rebooted the node. Then I've added new disks and had a look in command history and tried to use the old commands without checking how the devices are named. After the reboot the journal SSD for some reason changed name to sdc. I did all the same with the other two nodes including reboot etc. without the journal SSD changing name.