how to install ceph influx module?

K

kyriazis

Well-Known Member
Oct 28, 2019
98
6
48
Austin, TX
Hello,

I would like to enable influx data gathering for ceph. The ceph documentation that I found have instructions on how to enable it, but not how to install it; I can only assume that it installs by default.

However, when I try to enable the influx ceph module in a Proxmox cluster, I get:

Code: 
# ceph mgr module enable influx
Error ENOENT: module 'influx' reports that it cannot run on the active manager daemon: influxdb python module not found (pass --force to force enablement)
#

Is there some documentation on how to install the influx ceph module in a manner that Proxmox package updates will behave "nicely" (ie. update the influx module together with the rest of ceph)?

Thanks!

george
 
kyriazis said:
Hello,

I would like to enable influx data gathering for ceph. The ceph documentation that I found have instructions on how to enable it, but not how to install it; I can only assume that it installs by default.

However, when I try to enable the influx ceph module in a Proxmox cluster, I get:

Code: 
# ceph mgr module enable influx
Error ENOENT: module 'influx' reports that it cannot run on the active manager daemon: influxdb python module not found (pass --force to force enablement)
#

Is there some documentation on how to install the influx ceph module in a manner that Proxmox package updates will behave "nicely" (ie. update the influx module together with the rest of ceph)?

Thanks!

george
Click to expand...

I found this, so It seems its enabled and installed from GUI in proxmox there are 2 options see below

External Metric Server​




screenshot/gui-datacenter-metric-server-list.png
In Proxmox VE, you can define external metric servers, which will periodically receive various stats about your hosts, virtual guests and storages.
Currently supported are:
The external metric server definitions are saved in /etc/pve/status.cfg, and can be edited through the web interface.

Graphite server configuration​

screenshot/gui-datacenter-metric-server-graphite.png
The default port is set to 2003 and the default graphite path is proxmox.
By default, Proxmox VE sends the data over UDP, so the graphite server has to be configured to accept this. Here the maximum transmission unit (MTU) can be configured for environments not using the standard 1500 MTU.
You can also configure the plugin to use TCP. In order not to block the important pvestatd statistic collection daemon, a timeout is required to cope with network problems.

Influxdb plugin configuration​

screenshot/gui-datacenter-metric-server-influxdb.png
Proxmox VE sends the data over UDP, so the influxdb server has to be configured for this. The MTU can also be configured here, if necessary.
Here is an example configuration for influxdb (on your influxdb server):
[[udp]]
enabled = true
bind-address = "0.0.0.0:8089"
database = "proxmox"
batch-size = 1000
batch-timeout = "1s"
With this configuration, your server listens on all IP addresses on port 8089, and writes the data in the proxmox database
Alternatively, the plugin can be configured to use the http(s) API of InfluxDB 2.x. InfluxDB 1.8.x does contain a forwards compatible API endpoint for this v2 API.
To use it, set influxdbproto to http or https (depending on your configuration). By default, Proxmox VE uses the organization proxmox and the bucket/db proxmox (They can be set with the configuration organization and bucket respectively).
Since InfluxDB’s v2 API is only available with authentication, you have to generate a token that can write into the correct bucket and set it.
In the v2 compatible API of 1.8.x, you can use user:password as token (if required), and can omit the organization since that has no meaning in InfluxDB 1.x.
You can also set the HTTP Timeout (default is 1s) with the timeout setting, as well as the maximum batch size (default 25000000 bytes) with the max-body-size setting (this corresponds to the InfluxDB setting with the same name).


hope you find this helpful

Kind Regards,
Spiro
 

Attachments

  • 1649413147399.png
    1649413147399.png
    39.2 KB · Views: 11
Thanks, @Spirog.

What you are pointing enables the Proxmox influx integration, which gathers Proxmox-related statistics. I am interested in Ceph-specific stats, that are not gathered by the Proxmox metric server.

Thank you!

George
 
kyriazis said:
Thanks, @Spirog.

What you are pointing enables the Proxmox influx integration, which gathers Proxmox-related statistics. I am interested in Ceph-specific stats, that are not gathered by the Proxmox metric server.

Thank you!

George
Click to expand...
Ok on the ceph website, while reading from my understanding - it show you have to have an influxdb installed to use the module.
From their website.

The influx module continuously collects and sends time series data to an influxdb database.
 
I already have an influxdb database installed on a VM that I use for other data collecting. That's not the problem.

What is the issue, though, what's the best way to install the ceph influx module. My understanding is that Proxmox "slightly" modifies ceph and distributes the ceph packages from their repo as opposed to the Debian repos. First of all, I didn't find any additional information on the ceph website on how to install the influx package before using the "ceph mgr module enable influx" command. Second, if there is a package that is part of ceph that contains that module, how can I make sure that the package is in sync with the Proxmox ceph distribution?

Thank you!

George
 
Have you found a solution to this?

I am interested in pulling my Ceph data into Grafana via InfluxDB so that I can monitor OSD status. I currently have this up and running (minus Ceph) with the Metric Server enabled in Prox. But I have hit a wall with the "ceph mgr module enable influx" route that others have mentioned elsewhere.
 
ceph mgr module enable influx

It should be included as part of core-modules I think. I just rebuilt my mgr node last night, and found it working fine with no special packages when I failed back over to it.

I do remember having to run a bunch of commands to setup the destination. However I now see if you run the ceph dashboard on the manager, you can edit those values there.
 
Last edited:
I'm a liar, it was broken. Just fixed it.

apt install python3-influxdb
root@felis:~# ceph mgr module enable influx
Error ENOENT: module 'influx' reports that it cannot run on the active manager daemon: influxdb python module not found (pass --force to force enablement)

(re-start mgr in gui)

root@felis:~# ceph mgr module enable influx
module 'influx' is already enabled
root@felis:~# netstat -anlp | grep 8086
tcp 0 0 192.168.10.9:36202 192.168.10.32:8086 TIME_WAIT -
tcp 0 0 192.168.10.9:36204 192.168.10.32:8086 TIME_WAIT -
 
  • Like
Reactions: ozdjh
Still no dice for me - I've installed PIP & retrieved the influxdb module on all three of my Prox servers. Restarted managers multiple times.

I tried '-force' but it killed all three of my managers. They wouldn't come back up. So I trashed them and created three new.

I did manage to get the Ceph dashboard up, just to see if any of the modules would work. I did have to '-force' it, but it is working now and it didn't kill my managers thankfully.
 
Errr.. I didn't install the influxdb python module via pip, I installed it via apt. I don't know if it will notice the pip one.
 
thanks for the tip but...

Still no dice - back to the original error: MGR_MODULE_DEPENDENCY: Module 'influx' has failed dependency: influxdb python module not found

Very odd. At least I have the dashboard up and running. I'd prefer to have it all in Grafana but I'll take what I can get!
 
I had the same issue.
Did run `apt install python3-influxdb` - mgr module could not be enabled.

A reboot of the proxmox host did help for me though - afterwards:
`ceph mgr module enable telegraf` now says `module 'telegraf' is already enabled`

Seems to work.

EDIT: did not work - I missed that I enabled telegraf instead of influx in the last command, which I tried before..

EDIT2: I had to run apt install python3-influxdb on the active manager - then restart this manager - now it works `ceph mgr module enable influx`

Code: 
ceph mgr module ls
MODULE                           
balancer           on (always on)
crash              on (always on)
devicehealth       on (always on)
orchestrator       on (always on)
pg_autoscaler      on (always on)
progress           on (always on)
rbd_support        on (always on)
status             on (always on)
telemetry          on (always on)
volumes            on (always on)
dashboard          on           
influx             on           
iostat             on           
nfs                on           
restful            on           
telegraf           on           
alerts             -             
insights           -             
localpool          -             
mirroring          -             
osd_perf_query     -             
osd_support        -             
prometheus         -             
selftest           -             
snap_schedule      -             
stats              -             
test_orchestrator  -             
zabbix             -
 
Last edited:
You must log in or register to reply here.
Top Bottom