hello
I've checked other threads and ceph lists.
For some reason a ceph disk lists as on 2 different ceph storage.
I backed up, deleted and restored the lxc to zfs.
now on pve storage list contents the disk still shows on both places.
so i spent some hours trying to remove. not done yet and may end up deleting the pool in a day or 2 any way.
i thought i'd post this info
and that is where i left off.
I've checked other threads and ceph lists.
For some reason a ceph disk lists as on 2 different ceph storage.
I backed up, deleted and restored the lxc to zfs.
now on pve storage list contents the disk still shows on both places.
so i spent some hours trying to remove. not done yet and may end up deleting the pool in a day or 2 any way.
i thought i'd post this info
Code:
# rbd status -p ceph vm-213-disk-1
Watchers:
watcher=10.11.12.3:0/2474997724 client.78184312 cookie=18446462598732840963
# rbd info vm-213-disk-1 -p ceph
rbd image 'vm-213-disk-1':
size 101GiB in 25856 objects
order 22 (4MiB objects)
block_name_prefix: rbd_data.77ad306b8b4567
format: 2
features: layering
flags:
create_timestamp: Sat Aug 25 04:44:22 2018
# rados -p ceph listwatchers rbd_header.77ad306b8b4567
watcher=10.11.12.3:0/2474997724 client.78184312 cookie=18446462598732840963
# i tried removing the mon . did not fix, added it back
# rbd showmapped
id pool image snap device
... ceph vm-213-disk-1 - /dev/rbd0
1 ceph vm-213-disk-1 - /dev/rbd1
# this worked
rbd unmap /dev/rbd0
# rbd unmap /dev/rbd1
rbd: sysfs write failed
rbd: unmap failed: (16) Device or resource busy
# rbd showmapped
id pool image snap device
1 ceph vm-213-disk-1 - /dev/rbd1
cat /sys/kernel/debug/ceph/220b9a53-4556-48e3-a73c-28deff665e45.client78184312/osdc
REQUESTS 0 homeless 0
LINGER REQUESTS
18446462598732840963 osd9 13.d511aa64 13.264 [9,49,40]/9 [9,49,40]/9 e76294 rbd_header.77ad306b8b4567 0x20 2 WC/0
BACKOFFS
# I tried stopping osd9 . that did not fix. another osd showed
# cat /sys/kernel/debug/ceph/220b9a53-4556-48e3-a73c-28deff665e45.client78184312/osdc
REQUESTS 0 homeless 0
LINGER REQUESTS
18446462598732840963 osd49 13.d511aa64 13.264 [49,40]/49 [49,40]/49 e76296 rbd_header.77ad306b8b4567 0x20 3 WC/0
BACKOFFS
# started osd9 again after a few min
# cat /sys/kernel/debug/ceph/220b9a53-4556-48e3-a73c-28deff665e45.client78184312/osdc
REQUESTS 0 homeless 0
LINGER REQUESTS
18446462598732840963 osd9 13.d511aa64 13.264 [9,49,40]/9 [9,49,40]/9 e76298 rbd_header.77ad306b8b4567 0x20 4 WC/0
BACKOFFS
and that is where i left off.
Last edited: