[SOLVED] Journal was not prepared with ceph-disk

[Firm]

Hi,

I use Proxmox 4.2 and wonder if things are ok:

# pveceph createosd /dev/sdc -journal_dev /dev/sda3
create OSD on /dev/sdc (xfs)
using device '/dev/sda3' for journal
Caution: invalid backup GPT header, but valid main header; regenerating
backup header from main header.

****************************************************************************
Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
verification and recovery are STRONGLY recommended.
****************************************************************************
GPT data structures destroyed! You may now partition the disk using fdisk or
other utilities.
Creating new GPT entries.
The operation has completed successfully.
WARNING:ceph-disk:OSD will not be hot-swappable if journal is not the same device as the osd data
WARNING:ceph-disk:Old blkid does not support ID_PART_ENTRY_* fields, trying sgdisk; may not correctly identify ceph volumes with dmcrypt
WARNING:ceph-disk:Journal /dev/sda3 was not prepared with ceph-disk. Symlinking directly.
Setting name!
partNum is 0
REALLY setting name!
The operation has completed successfully.
meta-data=/dev/sdc1  isize=2048  agcount=4, agsize=244188597 blks
  =  sectsz=512  attr=2, projid32bit=1
  =  crc=0  finobt=0
data  =  bsize=4096  blocks=976754385, imaxpct=5
  =  sunit=0  swidth=0 blks
naming  =version 2  bsize=4096  ascii-ci=0 ftype=0
log  =internal log  bsize=4096  blocks=476930, version=2
  =  sectsz=512  sunit=0 blks, lazy-count=1
realtime =none  extsz=4096  blocks=0, rtextents=0
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot.
The operation has completed successfully.

Before reboot I have the following partitions on /dev/sda:

sda  8:0  0 186.3G  0 disk  
├─sda1  8:1  0  1M  0 part  
├─sda2  8:2  0  244M  0 part  
│ └─md0  9:0  0 243.8M  0 raid1 /boot
├─sda3  8:3  0  5G  0 part  
└─sda4  8:4  0 181.1G  0 part  
  └─md1  9:1  0  181G  0 raid1
  └─vg0-root 251:0  0  15.3G  0 lvm  /

But after reboot /dev/sda3 is missing:

sda  8:0  0 186.3G  0 disk  
├─sda1  8:1  0  1M  0 part  
├─sda2  8:2  0  244M  0 part  
│ └─md0  9:0  0 243.8M  0 raid1 /boot
└─sda4  8:4  0 181.1G  0 part  
  └─md1  9:1  0  181G  0 raid1
  └─vg0-root 251:0  0  15.3G  0 lvm  /

But osd.0 disk is up and running in Proxmox. And I have

ls -l /var/lib/ceph/osd/ceph-0/journal
lrwxrwxrwx 1 root root 9 Jun 28 18:00 /var/lib/ceph/osd/ceph-0/journal -> /dev/sda3

which looks broken as /dev/sda3 is missing.

Questions:
1. Should I recreate partition manually?
2. Even if I try to set journal partition GUID manually, name it 'ceph journal', use ceph-disk to create OSD I get the same warning.

In both cases (pveceph and ceph-disk) osd.0 appears in Proxmox UI.

P.S. It seems to me that this warning can be silently ignored in my case but want to be sure.

Regards,
Alex

 

[Firm]

Very strange/bad. Even running

ceph-disk prepare --fs-type xfs --cluster ceph --cluster-uuid 908ceb45-91b6-4c31-8ede-00acab17c9ef --journal-dev /dev/sdc /dev/sda3

deletes /dev/sda3

 

[Firm]

For those who are interested in creating osd journal on separate partition (not disk) here are the steps:
(assume that sda3 and sdb3 - 5GB partitions for journal, sdc and sdd - disks for osd data)

1. Create partition with correct size. If using fdisk partition's size should be 10483712 sectors.
2. Create osd disk with file-based journal

ceph-disk prepare --fs-type xfs --cluster ceph --cluster-uuid 908ceb45-91b6-4c31-8ede-00acab17c9ef --journal-file /dev/sdc /tmp/journal

3. Stop this osd.0 in Proxmox UI
4. Flush journal (as precaution)

ceph-osd --flush-journal -i 0

5. Relink journal to partition:

ln -sf /dev/sda3 /var/lib/ceph/osd/ceph-0/journal

6. Create journal

ceph-osd --mkjournal -i 0

7. Start osd.0 in Proxmox UI
8. Delete /tmp/journal and repeat steps above for sdb3 and sdd