[SOLVED] Proxmox prometheus exporter questions

[jfmanamtr]

I'm not sure if this is the right place to ask this or not. I have a 2 node proxmox cluster. I'm running Prometheus and Grafana on a vm. I want to use this to monitor the proxmox systems and report to me through slack.

I have the system setup and am trying to get the proxmox exporter exposing those metrics so I can scrape them with the server. When I try to look at the http://ip:9221/pve, I get the following: Couldn't authenticate user: prometheus@pve to https://localhost:8006/api2/json/access/ticket

I think this is something I'm doing wrong as far as the api. But I'm not sure. Has anyone seen anything like this before or can you point me in the right direction?

Thanks

 

[Dominic]

Hi!

Did you verify with /etc/pve/user.cfg if your users and realm (see documentation for details) are really correct? Have you had any success doing the authentication manually (using curl), as explained in the Wiki?

If you don't mind some dependencies, there are tools/guides available that might help. For example this article.

 

[jfmanamtr]

Hi!

Did you verify with /etc/pve/user.cfg if your users and realm (see documentation for details) are really correct? Have you had any success doing the authentication manually (using curl), as explained in the Wiki?

If you don't mind some dependencies, there are tools/guides available that might help. For example this article.

I am at work but will triple check and post here when I get home later. Thanks. I didn't try it manually yet.

 

[jfmanamtr]

ok. so, I think I have it all set right.

root@node-a:~# cat /etc/pve/user.cfg       

user:prometheus@pve:1:0::::::
token:prometheus@pve!ab27.....031e:0:0::
user:root@pam:1:0:::::

when I go to http://ip:9221/pve?target=ip (prometheus exporter), I get the following:

Couldn't authenticate user: prometheus@pve to https://ip:8006/api2/json/access/ticket

then when I go to https://ip:8006/api2/json/access/ticket, I get the following error:

{"data":null}

Any ideas? I will post anything you need to help me. Just let me know. Thanks

 

[Dominic]

then when I go to https://ip:8006/api2/json/access/ticket, I get the following error:

By "I got to", do you mean entering this into the browser? Try using the curl command from the Wiki instead.

 

[jfmanamtr]

Apologies. I forgot to add that part to my last reply.

root@node-a:~# curl -k -d "username=prometheus@pve&password=<redacted>"  https://node-a:8006/api2/json/access/ticket                       

{"data":{"username":"prometheus@pve","cap":{"nodes":{},"access":{},"dc":{},"vms":{},"storage":{}},"ticket":"PVE:prometheus@pve:5F4697F2::tZdQenzHv5dZ2nMEI0fHPdCtj8ZcJPEg6WDA9xPdUb7rwyEPp1nUk5vXzUgU+fkMMpITMH7fIuhJplD9FgY4c/2e+ihw27hAtCajYN/OhLC+hGALaDHv7gpFZz2fBLd0d0CLvK
InvlEFyzD3ogWf4M2Vu8PB5Esb8rvwk2gyF8qnmAZhpju5AebK3DgAnxQGKnVCzGa7iVRIJZDWqX2vmkJdu3HNDmlXTXew8oNNTyORf3M4c59Lu245HXG11/28yAxnwnn3ygWhdzFgAanqoQcW0HC0G6ifcY/LYZ9VFEcTdNTsypASjf7k9L1ZXQyViP9O3VwDNyASCTgzcKih1g==","CSRFPreventionToken":"5F4697F2:aeUa65cbg3rEncOHtX7ArpCjbA5

I am assuming this means that the API is working as expected then?

 

[jfmanamtr]

I think I got this figured out. I had to set permissions on the cluster. I believe I have this working now.

Give me a little time and I'll put together a synopsis of what I did cause the instructions I followed are severely lacking.

 

[jfmanamtr]

ok. So, I followed these instructions, but they are a little incomplete. Specifically the section marked:

Install proxmox-pve-exporter

There are some steps you have to do inside of Proxmox to make Prometheus pve exporter work correctly.

You have to create a read only user under the datacenter > users section, click add.



Screen looks like this. Fill in the user name, set realm to Proxmox VE authentication & give it your password & confirm password. Finally click add



Move to the api tokens section. select the user you just created & give it a token id. then finally click add.



lastly go back to the permissions. Set the path to / & select your user.



Under role, make sure to select PVE Auditor.


Then follow those steps for the promoetheus-pve-exporter & it should work.

 

[Dominic]

Thank you for posting the update!

 

[sshaikh]

@jfmanamtr this was great. Just out of interest, why do you create a token? It doesn't look like the pve exporter supports them? I had to use the user/pass.

 

[sshaikh]

From https://github.com/prometheus-pve/prometheus-pve-exporter/issues/43, you can use tokens by using a slightly different config:

default:
  user: "..."
  token_name: "..."
  token_value: "..."

 

[jfmanamtr]

From https://github.com/prometheus-pve/prometheus-pve-exporter/issues/43, you can use tokens by using a slightly different config:

default:
  user: "..."
  token_name: "..."
  token_value: "..."

Ah. I honestly just used the token cause it worked. No special reason for it over any other way.

 

[thantos71]

Thanks for this as it really helped!

I used the token method and I still had an issue with a 403 error.
The key was to untick the privilege separation box in the 4th screenshot where you add the token and then everything works great

 

[fabian]

Thanks for this as it really helped!

I used the token method and I still had an issue with a 403 error.
The key was to untick the privilege separation box in the 4th screenshot where you add the token and then everything works great

alternatively you could give the token the permissions it needs