Add OSD Ceph always out

[cheissol]

Hi,
I've got a cluster with 3 nodes. In node 2 I was upgrading 2 OSD of 4; the upgrade of these 2 osd was ok but one osd not updated was down/in during this upgrade. I waited until the rebalance was done and then from GUI I put the OSD out and destroy it. I thought that create it again was enought, but no. Now every time I try to recreate the OSD it is always put out:

~# ceph osd tree
ID CLASS WEIGHT  TYPE NAME      STATUS REWEIGHT PRI-AFF
-1       3.48570 root default                           
-2       0.92877     host vs1                           
 0   hdd 0.23219         osd.0      up  1.00000 1.00000
 1   hdd 0.23219         osd.1      up  1.00000 1.00000
 6   hdd 0.23219         osd.6      up  1.00000 1.00000
 9   hdd 0.23219         osd.9      up  1.00000 1.00000
-3       1.16237     host vs2                           
 2   hdd 0.23219         osd.2      up  1.00000 1.00000
 3   hdd 0.46509         osd.3      up  1.00000 1.00000
 7   hdd 0.46509         osd.7      up  1.00000 1.00000
-4       1.39456     host vs3                           
 4   hdd 0.23219         osd.4      up  1.00000 1.00000
 5   hdd 0.46509         osd.5      up  1.00000 1.00000
 8   hdd 0.46509         osd.8      up  1.00000 1.00000
11   hdd 0.23219         osd.11     up  1.00000 1.00000
10             0 osd.10           down        0 1.00000

I tried to delete osd.10 at all:

~# ceph osd crush remove osd.10
~# ceph auth del osd.10
updated
~# systemctl disable ceph-osd@osd.10
~# systemctl stop ceph-osd@osd.10
~# ceph osd rm osd.10
removed osd.10
~# ceph osd tree
ID CLASS WEIGHT  TYPE NAME      STATUS REWEIGHT PRI-AFF
-1       3.48570 root default                           
-2       0.92877     host vs1                           
 0   hdd 0.23219         osd.0      up  1.00000 1.00000
 1   hdd 0.23219         osd.1      up  1.00000 1.00000
 6   hdd 0.23219         osd.6      up  1.00000 1.00000
 9   hdd 0.23219         osd.9      up  1.00000 1.00000
-3       1.16237     host vs2                           
 2   hdd 0.23219         osd.2      up  1.00000 1.00000
 3   hdd 0.46509         osd.3      up  1.00000 1.00000
 7   hdd 0.46509         osd.7      up  1.00000 1.00000
-4       1.39456     host vs3                           
 4   hdd 0.23219         osd.4      up  1.00000 1.00000
 5   hdd 0.46509         osd.5      up  1.00000 1.00000
 8   hdd 0.46509         osd.8      up  1.00000 1.00000
11   hdd 0.23219         osd.11     up  1.00000 1.00000

And recreate OSD with GUI but nothing changes.

What can I do?
Thanks.
Cheis

 

[Alwin]

Overwrite the first 200 MB of the OSD with dd and then run a 'sgdisk -Z' on it. This happens sometimes with OSDs that already have been used before.