CEPH device removal

[andezett]

Im trying to remove the LVM from /dev/sda with no luck.


root@focuz:~# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 3.7T 0 disk
└─ceph--0e6896c9--c5c4--42f9--956e--177e173005ce-osd--block--fdcf2a33--ab58--4569--a79a--3b3ea336867f
253:0 0 3.7T 0 lvm
root@focuz:~# ceph-volume lvm zap /dev/sda --destroy
--> Zapping: /dev/sda
stderr: wipefs: error: /dev/sda: probing initialization failed: Device or resource busy
--> failed to wipefs device, will try again to workaround probable race condition

any suggestions ?

 

[t.lamprecht]

Can you please include some details, i.e., is this OSD still part of an (active?) ceph cluster?
Why do you want to remove it now, etc. That can help in providing a better solution.

It seems this is a still active OSD, you can check which programs have its block device open by executing:

lsof /dev/ceph-0e6896c9-c5c4-42f9-956e-177e173005ce/osd-block-fdcf2a33-ab58-4569-a79a-3b3ea336867f

If you now the OSD id (the integer after osd.) you could try to use pveceph osd destroy ID just replace ID with the desired one (and double check you got the right one if there are more than one)

 

[andezett]

Can you please include some details, i.e., is this OSD still part of an (active?) ceph cluster?
Why do you want to remove it now, etc. That can help in providing a better solution.

It seems this is a still active OSD, you can check which programs have its block device open by executing:

lsof /dev/ceph-0e6896c9-c5c4-42f9-956e-177e173005ce/osd-block-fdcf2a33-ab58-4569-a79a-3b3ea336867f

If you now the OSD id (the integer after osd.) you could try to use pveceph osd destroy ID just replace ID with the desired one (and double check you got the right one if there are more than one)

Thx for your reply Thomas.
i dont want to use ceph on my cluster the network is to slow and i did everyting to remove it rm files and kill procs.

lsof did not print any output and i dont find any OSD info everything seems to be removed.

 

[t.lamprecht]

OK, thanks for the additional info. Normally just running the pveceph destroy command should take care of all that (just FYI, for a potential next time).

Any how, zapping takes normally the partition, not the whole disk:

ceph-volume lvm zap --destroy /dev/ceph-0e6896c9-c5c4-42f9-956e-177e173005ce/osd-block-fdcf2a33-ab58-4569-a79a-3b3ea336867f

If that still fails then just use wipefs directly and tell it to force the wipe:

 # WARNING: data destroying potential!! Ensure /dev/sda is the device you want to destroy!!
wipefs -af /dev/sda
# additionally write the first 200 MB to zeros - should not really be required, so just to be sure™
dd if=/dev/zero bs=1M conv=fdatasync count=200 of=/dev/sda

 

[andezett]

OK, thanks for the additional info. Normally just running the pveceph destroy command should take care of all that (just FYI, for a potential next time).

Any how, zapping takes normally the partition, not the whole disk:

ceph-volume lvm zap --destroy /dev/ceph-0e6896c9-c5c4-42f9-956e-177e173005ce/osd-block-fdcf2a33-ab58-4569-a79a-3b3ea336867f

If that still fails then just use wipefs directly and tell it to force the wipe:

 # WARNING: data destroying potential!! Ensure /dev/sda is the device you want to destroy!!
wipefs -af /dev/sda
# additionally write the first 200 MB to zeros - should not really be required, so just to be sure™
dd if=/dev/zero bs=1M conv=fdatasync count=200 of=/dev/sda

root@focuz:~# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 3.7T 0 disk
└─ceph--0e6896c9--c5c4--42f9--956e--177e173005ce-osd--block--fdcf2a33--ab58--4569--a79a--3b3ea336867f
253:0 0 3.7T 0 lvm
i did all commands with no errors but still in kernel do i have to reboot to release it ?

 

[t.lamprecht]

i did all commands with no errors but still in kernel do i have to reboot to release it ?

Yeah, kernel partition tables need to be refreshed.

You can try the following first

partprobe /dev/sda

most of the time that is enough, else a reboot should handle it just fine.

 

[andezett]

thx thomas !
reboot did it.
BR
Anders

 

[ZKallo]

I've read this here in the forum but I don't know the author.
If you can't wipe the previously used OSD disk therefore you can't readd the disk in PVE8 do the following:
list the disk with
lsblk
You can see that the required disk was used for ceph because it has a ceph ID beginning with 'ceph-'
You can use this command:
dmsetup remove <ceph ID>
After that you can wipe the disk on GUI and the disk will appear at creating the OSD.

Best regards,
Zoltan