Ceph trying to reset class

DynFi User

Well-Known Member
Apr 18, 2016
147
16
58
48
dynfi.com
I have noticed these errors on my cluster (on every node for all my osd) :

Dec 20 22:00:28 pve1 ceph-osd[3991]: 2021-12-20T22:00:28.350+0100 7fbd20fa4f00 -1 osd.2 1257 mon_cmd_maybe_osd_create fail: 'osd.2 has already bound to class 'nvme', can not reset class to 'ssd'; use 'ceph osd crush rm-device-class <id>' to remove old class first': (16) Device or resource busy Dec 20 22:00:29 pve1 ceph-osd[4343]: 2021-12-20T22:00:29.558+0100 7f888a014f00 -1 osd.0 1257 mon_cmd_maybe_osd_create fail: 'osd.0 has already bound to class 'nvme', can not reset class to 'ssd'; use 'ceph osd crush rm-device-class <id>' to remove old class first': (16) Device or resource busy Dec 20 22:00:29 pve1 ceph-osd[4339]: 2021-12-20T22:00:29.782+0100 7f8de72b9f00 -1 osd.1 1257 mon_cmd_maybe_osd_create fail: 'osd.1 has already bound to class 'nvme', can not reset class to 'ssd'; use 'ceph osd crush rm-device-class <id>' to remove old class first': (16) Device or resource busy Dec 21 16:23:23 pve1 ceph-osd[3849]: 2021-12-21T16:23:23.855+0100 7f5f1eb30f00 -1 osd.0 1425 mon_cmd_maybe_osd_create fail: 'osd.0 has already bound to class 'nvme', can not reset class to 'ssd'; use 'ceph osd crush rm-device-class <id>' to remove old class first': (16) Device or resource busy Dec 21 16:23:44 pve1 ceph-osd[3843]: 2021-12-21T16:23:44.098+0100 7fe9e97e5f00 -1 osd.2 1425 mon_cmd_maybe_osd_create fail: 'osd.2 has already bound to class 'nvme', can not reset class to 'ssd'; use 'ceph osd crush rm-device-class <id>' to remove old class first': (16) Device or resource busy Dec 21 16:23:52 pve1 ceph-osd[3490]: 2021-12-21T16:23:52.169+0100 7feb5212af00 -1 osd.1 1425 mon_cmd_maybe_osd_create fail: 'osd.1 has already bound to class 'nvme', can not reset class to 'ssd'; use 'ceph osd crush rm-device-class <id>' to remove old class first': (16) Device or resource busy


I wanted to know if it was safe to proceed with the provided solution (also described here : https://ceph.io/en/news/blog/2017/new-luminous-crush-device-classes/ )

# ceph osd crush rm-device-class osd.0 osd.1 osd.2 # ceph osd crush set-device-class nvme osd.0 osd.1 osd.2


Knowing that my disks are already mapped to this class.
Somehow it looks like Ceph got confused and has kept both initial and new class.

Thanks for letting me know if this "ok".
 
This is working and seems to solve the pb.
 
  • Like
Reactions: aaron
Did not fix it for me, did you do anything else ? After rebooting ceph still brings up this error:


Code:
Mar 30 11:42:49 pmx01 ceph-osd[2900]: 2022-03-30T11:42:49.866+0200 7f49b844df00 -1 osd.2 932 mon_cmd_maybe_osd_create fail: 'osd.2 has already bound to class 'nvme', can not reset class to 'ssd'; use 'ceph osd crush rm-device-class <id>' to remove old class first': (16) Device or resource busy
Mar 30 11:43:00 pmx01 ceph-osd[2903]: 2022-03-30T11:43:00.543+0200 7fcb954e7f00 -1 osd.3 932 mon_cmd_maybe_osd_create fail: 'osd.3 has already bound to class 'nvme', can not reset class to 'ssd'; use 'ceph osd crush rm-device-class <id>' to remove old class first': (16) Device or resource busy
Mar 30 11:43:00 pmx01 ceph-osd[2875]: 2022-03-30T11:43:00.891+0200 7f3f5fd4af00 -1 osd.0 932 mon_cmd_maybe_osd_create fail: 'osd.0 has already bound to class 'nvme', can not reset class to 'ssd'; use 'ceph osd crush rm-device-class <id>' to remove old class first': (16) Device or resource busy
Mar 30 11:43:23 pmx01 ceph-osd[2873]: 2022-03-30T11:43:23.246+0200 7ffb75e5df00 -1 osd.1 932 mon_cmd_maybe_osd_create fail: 'osd.1 has already bound to class 'nvme', can not reset class to 'ssd'; use 'ceph osd crush rm-device-class <id>' to remove old class first': (16) Device or resource busy

did the two commands but error still on reboot:

Mar 30 14:14:05 pmx01 ceph-osd[2990]: 2022-03-30T14:14:05.297+0200 7f552a70bf00 -1 osd.0 981 mon_cmd_maybe_osd_create fail: 'osd.0 has already bound to class 'nvme', can not reset class to 'ssd'; use 'ceph osd crush rm-device-class <id>' to remove old class first': (16) Device or resource busy
Mar 30 14:14:05 pmx01 ceph-osd[2998]: 2022-03-30T14:14:05.401+0200 7f462718ff00 -1 osd.2 981 mon_cmd_maybe_osd_create fail: 'osd.2 has already bound to class 'nvme', can not reset class to 'ssd'; use 'ceph osd crush rm-device-class <id>' to remove old class first': (16) Device or resource busy
Mar 30 14:14:05 pmx01 ceph-osd[2987]: 2022-03-30T14:14:05.449+0200 7f85ad9f2f00 -1 osd.1 981 mon_cmd_maybe_osd_create fail: 'osd.1 has already bound to class 'nvme', can not reset class to 'ssd'; use 'ceph osd crush rm-device-class <id>' to remove old class first': (16) Device or resource busy
Mar 30 14:14:05 pmx01 ceph-osd[3004]: 2022-03-30T14:14:05.929+0200 7f4938e7df00 -1 osd.3 981 mon_cmd_maybe_osd_create fail: 'osd.3 has already bound to class 'nvme', can not reset class to 'ssd'; use 'ceph osd crush rm-device-class <id>' to remove old class first': (16) Device or resource busy
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!