[ceph] New /dev/sdX for active OSD

[Belokan]

Hello,

I have a 3 nodes PVE (4.4-18) with CEPH storage (10.2) configured.
I have to add a SATA controller and some disks/SSDs on one server.

Is it a problem if active OSDs have a new /dev/sdX during the process ? Or if an OSD located on /dev/sdf will be auto discovered if it is now /dev/sdh like md RAID does ?

Thanks in advance and have a nice weekend,

Olivier

 

[udo]

Hi,
no - the OSDs are mountet first on an temp-dir, read the osd-number on the osd (whoami) and remount the disk on the right place.

Udo

 

[Belokan]

Thanks for your answer.
Is there's a conf file somewhere this information is written ? So far I have:

mtab:/dev/sdf1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,noquota 0 0

Which is not in the fstab or any conf file. So I guess it is kind of discovered by a process started as a service ...

But even after installing the card and disks and /dev/sdf becomes /dev/sdg just mounting /dev/sdg1 on /var/lib/ceph/osd/ceph-0 will not make the "change" permanent right ?

 

[udo]

Thanks for your answer.
Is there's a conf file somewhere this information is written ?

Hi,
no - like I wrote before: ceph mount the osd on the right place due the info on the osd itself.

On an OSD are some files with the information for ceph to do the right thing:
Are the disk part of the cluster?
ceph_fsid: must contain the same fsid, like the cluster to join
Can ceph access the osd?
keyring: "password" to access the disk
Which osd-number this disk have?
whoami: the osd-number

cat /var/lib/ceph/osd/ceph-2/whoami
2

Udo

 

[Belokan]

So this means that all disks are scanned at startup in order to find if some are OSD ? This means that if /dev/sdf becomes /dev/sdg everything will work anyway ?

Sorry Udo ...