HowTo: Upgrade Ceph Hammer to Jewel

[tom]

Bump.. Anyone resolve this issue w/ Jewel 10.2.5?

as this problem does not exists on other places, something must be different in your cluster. dig deeper.

 

[blargling]

as this problem does not exists on other places, something must be different in your cluster. dig deeper.

I am also experiencing this error and have not found a solution. Looking ceph's code, I suspect Proxmox is trying to disable a rbd feature that is now a default in Jewel. Unfortunately rdb returns an error whenever something attempts to disabled a feature that is already disabled.

Specific, see here in ceph's code: https://github.com/ceph/ceph/blob/master/src/librbd/Operations.cc#L1279

What confuses me is that form the github blame it appears this error was not added until Ceph 11.x

 

[cloudguy]

When I upgraded remaining nodes to Jewel, the issue disappeared. Prior to this, I had one node upgraded, the remaining nodes on Hammer. I don't recall reading on the WIKI that all nodes must be at the same ceph release to create new RBD images.

 

[gkovacs]

We just published a new wiki page describing the upgrade from Ceph Hammer to Jewel (10.2.5 or higher). Before you start, make sure that you run very latest Proxmox VE 4.4.

The upgrade HowTo
https://pve.proxmox.com/wiki/Ceph_Hammer_to_Jewel

Doing the upgrade step by step. When trying to set the following permission, the command hangs on all nodes that have an OSD:

chown ceph: -R /var/lib/ceph/

I can't get past this command, despite that no OSD or MON processes run on the node. I have since stopped the entire Ceph cluster, but this command still won't complete on nodes that have an OSD.

Any idea how to proceed?

 

[wolfgang]

I can't get past this command, despite that no OSD or MON processes run on the node. I have since stopped the entire Ceph cluster, but this command still won't complete on nodes that have an OSD.

Your sure it it hanging?
This operation can take very long depends on the count, size and fill rate of the osd's .

 

[gkovacs]

Your sure it it hanging?
This operation can take very long depends on the count, size and fill rate of the osd's .

Upon further examination, it looks like it's running, albeit very slowly. Has taken more than 2 hours to complete on an SSD based RAIDZ. Thanks for your help.