hot to delete a Nonexistent OSD

[huky]

I added a ssd on host and wated convert the Journal filestore to bluestore
but after cmd on node006

pveceph osd create /dev/sdd -bluestore -wal_dev /dev/nvme0n1p1 -journal_dev /dev/nvme0n1p7

i got a Nonexistent OSD(osd.19) on node008

finally, I rebuild new osd via gui
but the Nonexistent OSD is still there:

root@node006:~# ceph osd tree
ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF
-1 77.86777 root default
-2 9.04999 host node001
0 hdd 1.81000 osd.0 up 1.00000 1.00000
4 hdd 1.81000 osd.4 up 1.00000 1.00000
6 hdd 1.81000 osd.6 up 1.00000 1.00000
9 hdd 1.81000 osd.9 up 1.00000 1.00000
12 hdd 1.81000 osd.12 up 1.00000 1.00000
-3 10.85999 host node002
1 hdd 1.81000 osd.1 up 1.00000 1.00000
3 hdd 1.81000 osd.3 up 1.00000 1.00000
7 hdd 1.81000 osd.7 up 1.00000 1.00000
10 hdd 1.81000 osd.10 up 1.00000 1.00000
13 hdd 1.81000 osd.13 up 1.00000 1.00000
16 hdd 1.81000 osd.16 up 1.00000 1.00000
-4 14.47998 host node003
2 hdd 1.81000 osd.2 up 1.00000 1.00000
5 hdd 1.81000 osd.5 up 1.00000 1.00000
8 hdd 1.81000 osd.8 up 1.00000 1.00000
11 hdd 1.81000 osd.11 up 1.00000 1.00000
14 hdd 1.81000 osd.14 up 1.00000 1.00000
17 hdd 1.81000 osd.17 up 1.00000 1.00000
20 hdd 1.81000 osd.20 up 1.00000 1.00000
23 hdd 1.81000 osd.23 up 1.00000 1.00000
-5 14.47998 host node004
27 hdd 1.81000 osd.27 up 1.00000 1.00000
29 hdd 1.81000 osd.29 up 1.00000 1.00000
33 hdd 1.81000 osd.33 up 1.00000 1.00000
34 hdd 1.81000 osd.34 up 1.00000 1.00000
35 hdd 1.81000 osd.35 up 1.00000 1.00000
36 hdd 1.81000 osd.36 up 1.00000 1.00000
41 hdd 1.81000 osd.41 up 1.00000 1.00000
42 hdd 1.81000 osd.42 up 1.00000 1.00000
-6 12.66998 host node005
28 hdd 1.81000 osd.28 up 1.00000 1.00000
31 hdd 1.81000 osd.31 up 1.00000 1.00000
37 hdd 1.81000 osd.37 up 1.00000 1.00000
38 hdd 1.81000 osd.38 up 1.00000 1.00000
39 hdd 1.81000 osd.39 up 1.00000 1.00000
40 hdd 1.81000 osd.40 up 1.00000 1.00000
45 hdd 1.81000 osd.45 up 1.00000 1.00000
-7 12.68927 host node006
21 hdd 1.81929 osd.21 up 1.00000 1.00000
25 hdd 1.81000 osd.25 up 1.00000 1.00000
30 hdd 1.81000 osd.30 up 1.00000 1.00000
32 hdd 1.81000 osd.32 up 1.00000 1.00000
43 hdd 1.81000 osd.43 up 1.00000 1.00000
47 hdd 3.62999 osd.47 up 1.00000 1.00000
-8 3.63858 host node008
15 hdd 1.81929 osd.15 up 1.00000 1.00000
18 hdd 1.81929 osd.18 up 1.00000 1.00000
19 0 osd.19 destroyed 0 1.00000

how could i delete it?

 

[Alwin]

OSD.19 is not really located on host node008, in the process of creating the OSD, something failed and that is the leftover.

If the OSD really doesn't exist, run the following:

ceph osd crush rm <ID>
ceph osd rm <ID>
ceph auth del osd.<ID>

 

[huky]

thanks, it's 'ceph osd rm 19'

root@node006:~# ceph osd crush rm 19
device '19' does not appear in the crush map
root@node006:~# ceph osd rm 19
removed osd.19
root@node006:~# ceph auth del osd.19
entity osd.19 does not exist

OSD.19 is not really located on host node008, in the process of creating the OSD, something failed and that is the leftover.

If the OSD really doesn't exist, run the following:

ceph osd crush rm <ID>
ceph osd rm <ID>
ceph auth del osd.<ID>