[SOLVED] osd blacklist spam on external ceph cluster

[qloc]

We are using Proxmox together with Ceph, with Ceph not running on the Proxmox hosts but on dedicated ones.

Since switching to luminous (and taking a look at the Audit log in the new dashboard) we are seeing the following spam by one of our five proxmox hosts:

2017-09-14 11:33:30.227695 [INF]  from='client.? 192.168.44.148:0/3594392926' entity='client.proxmox' cmd=[{"prefix": "osd blacklist", "blacklistop": "add", "addr": "192.168.44.148:0/3139463177"}]: access denied
2017-09-14 11:33:30.224874 [INF]  from='client.? 192.168.44.148:0/3594392926' entity='client.proxmox' cmd=[{"prefix": "osd blacklist", "blacklistop": "add", "addr": "192.168.44.148:0/3139463177"}]: access denied
2017-09-14 11:33:30.220335 [INF]  from='client.? 192.168.44.148:0/3594392926' entity='client.proxmox' cmd=[{"prefix": "osd blacklist", "blacklistop": "add", "addr": "192.168.44.148:0/3139463177"}]: access denied
2017-09-14 11:33:30.217575 [INF]  from='client.? 192.168.44.148:0/3594392926' entity='client.proxmox' cmd=[{"prefix": "osd blacklist", "blacklistop": "add", "addr": "192.168.44.148:0/3139463177"}]: access denied
2017-09-14 11:33:30.214310 [INF]  from='client.? 192.168.44.148:0/3594392926' entity='client.proxmox' cmd=[{"prefix": "osd blacklist", "blacklistop": "add", "addr": "192.168.44.148:0/3139463177"}]: access denied
2017-09-14 11:33:30.211594 [INF]  from='client.? 192.168.44.148:0/3594392926' entity='client.proxmox' cmd=[{"prefix": "osd blacklist", "blacklistop": "add", "addr": "192.168.44.148:0/3139463177"}]: access denied
2017-09-14 11:33:30.208554 [INF]  from='client.? 192.168.44.148:0/3594392926' entity='client.proxmox' cmd=[{"prefix": "osd blacklist", "blacklistop": "add", "addr": "192.168.44.148:0/3139463177"}]: access denied
2017-09-14 11:33:30.205559 [INF]  from='client.? 192.168.44.148:0/3594392926' entity='client.proxmox' cmd=[{"prefix": "osd blacklist", "blacklistop": "add", "addr": "192.168.44.148:0/3139463177"}]: access denied
2017-09-14 11:33:30.202905 [INF]  from='client.? 192.168.44.148:0/3594392926' entity='client.proxmox' cmd=[{"prefix": "osd blacklist", "blacklistop": "add", "addr": "192.168.44.148:0/3139463177"}]: access denied
2017-09-14 11:33:30.200070 [INF]  from='client.? 192.168.44.148:0/3594392926' entity='client.proxmox' cmd=[{"prefix": "osd blacklist", "blacklistop": "add", "addr": "192.168.44.148:0/3139463177"}]: access denied
2017-09-14 11:33:30.197138 [INF]  from='client.? 192.168.44.148:0/3594392926' entity='client.proxmox' cmd=[{"prefix": "osd blacklist", "blacklistop": "add", "addr": "192.168.44.148:0/3139463177"}]: access denied

As you can see by the IP, it's only one of the hosts thats trying to blacklist (itself?) - is it supposed to be this way and if so, which permissions does the key need?

 

[fabian]

possibly http://tracker.ceph.com/issues/21353 ? I recommend checking the capabilities of the keys you have configured to access your storages, and updating them if necessary.

 

[qloc]

Thanks, I updated my keys with the "profile rbd" entry for mon and osd.

Ceph still shows "access denied" for the blacklist command - do I need to restart my proxmox servers for the updated key to be used (just replaced the old one) or is there an easier way?

EDIT: The spam is gone, maybe it just takes a while for the permissions to update