Showing extra OSD

[ChrisJM]

Hello.

We have had some issues with some SSD disks and have had to remove a OSD and replace the disk then add it back in.

we have 3 nodes each with 3 OSD's = 9 in total but it is showing 10 and 1 out.

https://screenshots.firefox.com/AmNplUlnrAA58S9D/10.9.0.195

Any ideas why that might be? or how to remove it - it is also stuck on active + clean where as it never used to be.

 

[Alwin]

The old OSD needs to be destroyed, see 'pveceph osd destroy': https://pve.proxmox.com/pve-docs/pveceph.1.html

 

[ChrisJM]

The thing is it has no ID, i am unable to find it.

 

[sb-jw]

Check all OSD services on the nodes, eventually you can find it this way.

 

[Alwin]

The thing is it has no ID, i am unable to find it.

Try with 'ceph osd tree', it should show up there.

 

[ChrisJM]

found it

root@INETC1083:~# ceph osd tree
ID CLASS WEIGHT  TYPE NAME          STATUS REWEIGHT PRI-AFF
-1       8.18633 root default
-3       2.72878     host INETC1083
 1   ssd 0.90959         osd.1          up  1.00000 1.00000
 6   ssd 0.90959         osd.6          up  1.00000 1.00000
 7   ssd 0.90959         osd.7          up  1.00000 1.00000
-5       2.72878     host INETC1084
 2   ssd 0.90959         osd.2          up  1.00000 1.00000
 3   ssd 0.90959         osd.3          up  1.00000 1.00000
 8   ssd 0.90959         osd.8          up  1.00000 1.00000
-7       2.72878     host INETC1085
 4   ssd 0.90959         osd.4          up  1.00000 1.00000
 5   ssd 0.90959         osd.5          up  1.00000 1.00000
 9   ssd 0.90959         osd.9          up  1.00000 1.00000
 0             0 osd.0                down        0 1.00000

0 0 osd.0 down 0 1.00000

So how would i go about removing it. would this command work

pveceph osd destroy 0

 

[Alwin]

So how would i go about removing it. would this command work
pveceph osd destroy 0

If it doesn't than you need to use the ceph tools.

ceph osd rm osd.0
ceph osd crush rm osd.0
ceph auth del osd.0