[SOLVED] ceph class issue with nautilus nvme p3700

RobFantini

Famous Member
May 24, 2012
2,008
102
133
Boston,Mass
I have a class set up for nvme.

that worked fine with pve5 / luminous for our intel nvme p3700's.

now when i add a nvme :

1- pve screen give warning about raid controller: Note: Ceph is not compatible with disks backed by a hardware RAID controller. For details see the reference documentation.
I forget if it did so for ssd's connected to our IT mode LSI HBA's

2- the nvme gets added to ssd class.

any suggestions?
 
Last edited:
here is part of tree:

more info
Code:
# ceph device ls|grep  nvme
INTEL_SSDPEDMD020T4D_HHHL_NVMe_2000GB_CVFT5190000Q2P0EGN pve14:nvme0n1 osd.4                 
INTEL_SSDPEDMD020T4D_HHHL_NVMe_2000GB_CVFT735300072P0OGN pve10:nvme0n1 osd.1                 
INTEL_SSDPEDMD020T4D_HHHL_NVMe_2000GB_CVFT7353000B2P0OGN pve3:nvme0n1  osd.2                 
INTEL_SSDPEDMD020T4D_HHHL_NVMe_2000GB_CVFT735300122P0OGN pve2:nvme0n1  osd.0                 
INTEL_SSDPEDMD020T4D_HHHL_NVMe_2000GB_CVFT735300282P0OGN pve9:nvme0n1  osd.6                 
INTEL_SSDPEDMD020T4_CVFT629000L92P0EGN                   sys8:nvme0n1  osd.3                 
INTEL_SSDPEDMD020T4_PHFT6201004W2P0EGN                   pve15:nvme0n1 osd.5
 
Last edited:
solved with info from https://access.redhat.com/solutions/3341491
Code:
## Initial members in nvme class
# ceph osd tree|grep nvme
  1  nvme  1.81929         osd.1      up  1.00000 1.00000
  4  nvme  1.81929         osd.4      up  1.00000 1.00000


 # ceph osd crush set-device-class nvme osd.0
Error EBUSY: osd.0 has already bound to class 'ssd', can not reset class to 'nvme'; use 'ceph osd crush rm-device-class <id>' to remove old class first

# ceph osd crush rm-device-class osd.0
done removing class of osd(s): 0

# ceph osd crush set-device-class nvme osd.0
set osd(s) 0 to class 'nvme'

## Result
# ceph osd tree|grep nvme
  1  nvme  1.81929         osd.1      up  1.00000 1.00000
  4  nvme  1.81929         osd.4      up  1.00000 1.00000
  0  nvme  1.81929         osd.0      up  1.00000 1.00000

after that ceph has some backfilling to do. so naturally follow the one thing at a time - let ceph -s show no backfills and do the next changes.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!