[SOLVED] Prometheus output empty after upgrade

Mikepop

Well-Known Member
Feb 6, 2018
63
6
48
51
Hi, I use a grafana multicluster long time ago and after upgrade I see an empty answer to the queries. Any idea how to debug this?

Code:
root@int101:~# ceph config dump
WHO    MASK  LEVEL     OPTION                                  VALUE          RO
  mon        advanced  auth_allow_insecure_global_id_reclaim   false           
  mgr        advanced  mgr/dashboard/FEATURE_TOGGLE_CEPHFS     false          *
  mgr        advanced  mgr/dashboard/FEATURE_TOGGLE_ISCSI      false          *
  mgr        advanced  mgr/dashboard/FEATURE_TOGGLE_MIRRORING  false          *
  mgr        advanced  mgr/dashboard/FEATURE_TOGGLE_RGW        false          *
  mgr        advanced  mgr/dashboard/int101/server_addr        10.62.91.101   *
  mgr        advanced  mgr/dashboard/int102/server_addr        10.62.91.102   *
  mgr        advanced  mgr/dashboard/int103/server_addr        10.62.91.103   *
  mgr        advanced  mgr/dashboard/int105/server_addr        10.62.91.105   *
  mgr        advanced  mgr/dashboard/ssl                       false          *
  mgr        advanced  mgr/prometheus/scrape_interval          20.000000      *
  mgr        advanced  mgr/prometheus/server_addr              185.18.17.102  *
  mgr        advanced  mgr/prometheus/server_port              9289           *
root@int101:~#
root@int101:~# curl 185.18.17.102:9289
<!DOCTYPE html>
<html>
    <head><title>Ceph Exporter</title></head>
    <body>
        <h1>Ceph Exporter</h1>
        <p><a href='http://185.18.17.102:9289/metrics'>Metrics</a></p>
    </body>
</html>
root@int101:~# curl 185.18.17.102:9289/metrics
root@int101:~#

Sep 29 17:08:21 int101 ceph-mgr[518994]: 185.18.17.102 - - [29/Sep/2021:17:08:21] "GET /metrics HTTP/1.1" 200 - "" "curl/7.74.0"

Regards
 
Nobody can suggest how to debug? I guess I'll have to try on ceph mail list...
I'll post the solution to the issue if I can solve it..


Regards
 
This is a conflint with dashboard, to solve it just run
Code:
ceph mgr module disable dashboard
Regards