Proxmox 5.0: Add Ceph RBD (external) running Ceph jewel or hammer

[Blinkiz]

Hello
Is it possible to add a Ceph RBD external storage that is running a version of Ceph that is lower than the Ceph client installed in Proxmox?

For example
If am running Proxmox 5.0 and want to add RBD (external) that is running Ceph Jewel, is this possible?
If am running Proxmox 5.0 and want to add RBD (external) that is running Ceph hammer, is this possible?

 

[Alwin]

As far as I know, yes, but you can only use features of the oldest client accessing the rbd.

 

[Blinkiz]

When adding a Ceph RBD external storage, I can see that Proxmox is trying to run

/usr/bin/rados -p default -m 10.100.1.1 --auth_supported cephx -n client.admin --keyring /etc/pve/priv/ceph/old.keyring df

Which result in CPU load and nothing else. Running the command manually I can see spamming of the line

7f622664d700 -1 failed to decode message of type 59 v1: buffer::end_of_buffer
7f622664d700 -1 failed to decode message of type 59 v1: buffer::end_of_buffer
7f622664d700 -1 failed to decode message of type 59 v1: buffer::end_of_buffer
7f622664d700 -1 failed to decode message of type 59 v1: buffer::end_of_buffer
7f622664d700 -1 failed to decode message of type 59 v1: buffer::end_of_buffer
7f622664d700 -1 failed to decode message of type 59 v1: buffer::end_of_buffer
...

Here is Proxmox 5.0 with ceph version 12.2.1 trying to communicate with a ceph version 0.94.3 (hammer).

Running other types of commands like lspools, ls and lssnap work so the communication is okay. Just the "df" command result in a error.

 

[aderumier]

Normally, ceph client should be backward compatible, but I'm not sure that ceph devs test all version.

externe Jewel is working fine, with librbd jewel or luminious on proxmox 5.

Don't have hammer to test.

maybe try to ask to the ceph dev mailing list ? it could be a bug

 

[Hans van den Bogert]

Im seeing similar errors, though with Proxmox 4 and an external Luminous cluster.

 

[Alwin]

Ceph hammer might be one LTS to far away for being tested.

@Hans van den Bogert, you might need to update your ceph packages too.

 

[Hans van den Bogert]

/new
Nevermind, I used the `pveceph` tool to upgrade.

/old

@Hans van den Bogert, you might need to update your ceph packages too.

You mean update the Ceph packages on the proxmox nodes? Is there an howto for that? Not that I'm unfamiliar with apt-get or anything, but I'd rather replay somebody else's tested commands to update Ceph in this case.