Minimal capabilities/permissions for ceph rbd storage?

[ermitgilsukaru]

Hi.

We have a ceph cluster at my work which we are mostly using for cephfs. I'd like to add it as a rbd target to a proxmox server I have, but I'd like to have lower permissions than using the client.admin keyring from the ceph cluster. I tried creating a new keyring with the following permissions:

client.proxmox.rbd
    key: xxxxxxxxxxxxxxxxxxxxx
    caps: [mgr] profile rbd pool=repl_pool, profile rbd pool=ec_pool
    caps: [mon] profile rbd
    caps: [osd] profile rbd pool=repl_pool, profile rbd pool=ec_pool

But if I try to list rbd images with the secret for this keyring I get errors like this:

[root@rl8-template ~]# rbd -K ~/client.proxmox.rbd.secret ls
2022-06-14T18:19:41.341+0000 7f3a74dc8700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2,1]
2022-06-14T18:19:41.342+0000 7f3a745c7700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2,1]
2022-06-14T18:19:41.342+0000 7f3a755c9700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2,1]
rbd: couldn't connect to the cluster!
rbd: listing images failed: (13) Permission denied

Listing images with the client.admin keyring works.

Does anybody know what minimal permissions I'd need to add to be able to use this keyring in proxmox?

Edit: forgot to add that when I tried adding the ceph cluster as a datastore in the proxmox webgui I get this error:

rbd error: rbd: listing images failed: (95) Operation not supported (500)

 

[dcsapak]

is it a pacific cluster but you use the debian default ceph packages?
i found this: https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/K7LDVS7Y5XQV7ILHC5WUWMXVJ5HX4HU3/
which suggests that you should update the client packages

EDIT: mhmm.. i didn't completely read the thread there, so it's probably not that problem
any logs from the ceph cluster maybe?

 

[ermitgilsukaru]

Thanks for the answer dcsapak!

is it a pacific cluster but you use the debian default ceph packages?
i found this: https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/K7LDVS7Y5XQV7ILHC5WUWMXVJ5HX4HU3/
which suggests that you should update the client packages

One of the ceph mon nodes shows this:

root@jarn33:~# ceph --version
ceph version 15.2.16 (d46a73d6d0a67a79558054a3a5a72cb561724974) octopus (stable)

...and the proxmox machine in question shows this:

root@pve2:~# ceph --version
ceph version 15.2.16 (a6b69e817d6c9e6f02d0a7ac3043ba9cdbda1bdf) octopus (stable)

So I would guess that it isn't a version mismatch

EDIT: mhmm.. i didn't completely read the thread there, so it's probably not that problem
any logs from the ceph cluster maybe?

Hmmm... Have to look into how to log authentication requests on the ceph side.