how to install ceph influx module?

[kyriazis]

Hello,

I would like to enable influx data gathering for ceph. The ceph documentation that I found have instructions on how to enable it, but not how to install it; I can only assume that it installs by default.

However, when I try to enable the influx ceph module in a Proxmox cluster, I get:

# ceph mgr module enable influx
Error ENOENT: module 'influx' reports that it cannot run on the active manager daemon: influxdb python module not found (pass --force to force enablement)
#

Is there some documentation on how to install the influx ceph module in a manner that Proxmox package updates will behave "nicely" (ie. update the influx module together with the rest of ceph)?

Thanks!

george

 

[Spirog]

Hello,

I would like to enable influx data gathering for ceph. The ceph documentation that I found have instructions on how to enable it, but not how to install it; I can only assume that it installs by default.

However, when I try to enable the influx ceph module in a Proxmox cluster, I get:

# ceph mgr module enable influx
Error ENOENT: module 'influx' reports that it cannot run on the active manager daemon: influxdb python module not found (pass --force to force enablement)
#

Is there some documentation on how to install the influx ceph module in a manner that Proxmox package updates will behave "nicely" (ie. update the influx module together with the rest of ceph)?

Thanks!

george

I found this, so It seems its enabled and installed from GUI in proxmox there are 2 options see below

External Metric Server​

In Proxmox VE, you can define external metric servers, which will periodically receive various stats about your hosts, virtual guests and storages.
Currently supported are:

• Graphite (see https://graphiteapp.org )
• InfluxDB (see https://www.influxdata.com/time-series-platform/influxdb/ )

The external metric server definitions are saved in /etc/pve/status.cfg, and can be edited through the web interface.

Graphite server configuration​

The default port is set to 2003 and the default graphite path is proxmox.
By default, Proxmox VE sends the data over UDP, so the graphite server has to be configured to accept this. Here the maximum transmission unit (MTU) can be configured for environments not using the standard 1500 MTU.
You can also configure the plugin to use TCP. In order not to block the important pvestatd statistic collection daemon, a timeout is required to cope with network problems.

Influxdb plugin configuration​

Proxmox VE sends the data over UDP, so the influxdb server has to be configured for this. The MTU can also be configured here, if necessary.
Here is an example configuration for influxdb (on your influxdb server):
[[udp]]
enabled = true
bind-address = "0.0.0.0:8089"
database = "proxmox"
batch-size = 1000
batch-timeout = "1s"
With this configuration, your server listens on all IP addresses on port 8089, and writes the data in the proxmox database
Alternatively, the plugin can be configured to use the http(s) API of InfluxDB 2.x. InfluxDB 1.8.x does contain a forwards compatible API endpoint for this v2 API.
To use it, set influxdbproto to http or https (depending on your configuration). By default, Proxmox VE uses the organization proxmox and the bucket/db proxmox (They can be set with the configuration organization and bucket respectively).
Since InfluxDB’s v2 API is only available with authentication, you have to generate a token that can write into the correct bucket and set it.
In the v2 compatible API of 1.8.x, you can use userassword as token (if required), and can omit the organization since that has no meaning in InfluxDB 1.x.
You can also set the HTTP Timeout (default is 1s) with the timeout setting, as well as the maximum batch size (default 25000000 bytes) with the max-body-size setting (this corresponds to the InfluxDB setting with the same name).


hope you find this helpful

Kind Regards,
Spiro

 

[kyriazis]

Thanks, @Spirog.

What you are pointing enables the Proxmox influx integration, which gathers Proxmox-related statistics. I am interested in Ceph-specific stats, that are not gathered by the Proxmox metric server.

Thank you!

George

 

[Spirog]

Thanks, @Spirog.

What you are pointing enables the Proxmox influx integration, which gathers Proxmox-related statistics. I am interested in Ceph-specific stats, that are not gathered by the Proxmox metric server.

Thank you!

George

Ok on the ceph website, while reading from my understanding - it show you have to have an influxdb installed to use the module.
From their website.

The influx module continuously collects and sends time series data to an influxdb database.

 

[kyriazis]

I already have an influxdb database installed on a VM that I use for other data collecting. That's not the problem.

What is the issue, though, what's the best way to install the ceph influx module. My understanding is that Proxmox "slightly" modifies ceph and distributes the ceph packages from their repo as opposed to the Debian repos. First of all, I didn't find any additional information on the ceph website on how to install the influx package before using the "ceph mgr module enable influx" command. Second, if there is a package that is part of ceph that contains that module, how can I make sure that the package is in sync with the Proxmox ceph distribution?

Thank you!

George

 

[b.miller]

Have you found a solution to this?

I am interested in pulling my Ceph data into Grafana via InfluxDB so that I can monitor OSD status. I currently have this up and running (minus Ceph) with the Metric Server enabled in Prox. But I have hit a wall with the "ceph mgr module enable influx" route that others have mentioned elsewhere.

 

[garbled]

ceph mgr module enable influx

It should be included as part of core-modules I think. I just rebuilt my mgr node last night, and found it working fine with no special packages when I failed back over to it.

I do remember having to run a bunch of commands to setup the destination. However I now see if you run the ceph dashboard on the manager, you can edit those values there.

 

[garbled]

I'm a liar, it was broken. Just fixed it.

apt install python3-influxdb
root@felis:~# ceph mgr module enable influx
Error ENOENT: module 'influx' reports that it cannot run on the active manager daemon: influxdb python module not found (pass --force to force enablement)

(re-start mgr in gui)

root@felis:~# ceph mgr module enable influx
module 'influx' is already enabled
root@felis:~# netstat -anlp | grep 8086
tcp 0 0 192.168.10.9:36202 192.168.10.32:8086 TIME_WAIT -
tcp 0 0 192.168.10.9:36204 192.168.10.32:8086 TIME_WAIT -

 

[b.miller]

Still no dice for me - I've installed PIP & retrieved the influxdb module on all three of my Prox servers. Restarted managers multiple times.

I tried '-force' but it killed all three of my managers. They wouldn't come back up. So I trashed them and created three new.

I did manage to get the Ceph dashboard up, just to see if any of the modules would work. I did have to '-force' it, but it is working now and it didn't kill my managers thankfully.

 

[garbled]

Errr.. I didn't install the influxdb python module via pip, I installed it via apt. I don't know if it will notice the pip one.

 

[b.miller]

thanks for the tip but...

Still no dice - back to the original error: MGR_MODULE_DEPENDENCY: Module 'influx' has failed dependency: influxdb python module not found

Very odd. At least I have the dashboard up and running. I'd prefer to have it all in Grafana but I'll take what I can get!