[SOLVED] ceph zabbix send Fehler nach Update auf PVE 6.1

Ingo S

Renowned Member
Oct 16, 2016
333
38
68
41
Hallo zusammen

Wir haben Zabbix als Monitoring und bislang erfolgreich auch die Ceph Metriken via dem zabbix Modul des Ceph MGR an unser Monitoring versendet. Gestern haben wir von PVE 6.0 auf 6.1 upgedated und seitdem meldet Ceph Failed to send data to Zabbix

Manuell erhalte ich folgende Meldungen auf der Konsole und im Log:

Bash:
root@vm-2:~# ceph zabbix send
Failed to send data to Zabbix
root@vm-2:~#
-----------
parallel dazu:
root@vm-2:~#journalctl -f
Dec 19 11:11:31 vm-2 ceph-mgr[2825]: 2019-12-19 11:11:31.870 7f2024255700 -1 mgr.server reply reply (1) Operation not permitted
Jemand ne Idee? Es wurde nichts an der Konfiguration geändert, wir haben nur die aktuellen Updates eingespielt.
 
Indeed, that solved the problem. I changed the Key ceph.num_pg_wait_backfill to ceph.num_pg_backfill_wait in the Template. A few minutes later the error message was gone.
Thx
 
  • Like
Reactions: klowet
Thank you for opening this thread. Unfortunately, i still have a problem even with right syntax in the zabbix_template.xml
any help please?


root@1:/usr/share/ceph/mgr/zabbix# grep ceph.num_pg_wait_backfill *.xml
root@1:/usr/share/ceph/mgr/zabbix# grep ceph.num_pg_backfill_wait *
zabbix_template.xml: <key>ceph.num_pg_backfill_wait</key>
root@1:/usr/share/ceph/mgr/zabbix#
ceph zabbix send -vv
ceph version 14.2.9 (bed944f8c45b9c98485e99b70e11bbcec6f6659a) nautilus (stable)
 
Hi sunkroot

on our server the search for "ceph.num_pg_wait_backfill" gives the exact same result. ceph.num_pg_backfill_wait * is present in zabbix_template.xml.
ceph zabbix send is working just fine and returns with "ceph version 14.2.6 (ba51347bdbe28c7c0e2e9172fa2983111137bb60) nautilus (stable)"

So, i don't know what the exact problem is on your side.
 
Hi sunkroot

on our server the search for "ceph.num_pg_wait_backfill" gives the exact same result. ceph.num_pg_backfill_wait * is present in zabbix_template.xml.
ceph zabbix send is working just fine and returns with "ceph version 14.2.6 (ba51347bdbe28c7c0e2e9172fa2983111137bb60) nautilus (stable)"

So, i don't know what the exact problem is on your side.

Thank you for looking into this issue. Unfortunately nothing from ceph app is send to zabbix server. I'm not sure where to begin..

root@pve1:/var/log/ceph# ceph zabbix send
Failed to send data to Zabbix
root@pve1:/var/log/ceph#

root@pve1:/var/log/ceph# ceph -v
ceph version 14.2.9 (bed944f8c45b9c98485e99b70e11bbcec6f6659a) nautilus (stable)
root@pve1:/var/log/ceph#

ceph-mgr.pve1.log:RuntimeError: /usr/bin/zabbix_sender exited non-zero: zabbix_sender [1994910]: DEBUG: answer [{"response":"success","info":"processed: 0; failed: 44; total: 44; seconds spent: 0.000449"}]
ceph-mgr.pve1.log:2020-04-27 10:29:05.598 7f7ea9b44700 0 mgr[zabbix] Failed to send.
ceph-mgr.pve1.log: File "/usr/share/ceph/mgr/zabbix/module.py", line 303, in send
ceph-mgr.pve1.log: File "/usr/share/ceph/mgr/zabbix/module.py", line 46, in send
ceph-mgr.pve1.log:RuntimeError: /usr/bin/zabbix_sender exited non-zero: zabbix_sender [2012205]: DEBUG: answer [{"response":"success","info":"processed: 0; failed: 44; total: 44; seconds spent: 0.000417"}]
root@pve1:/var/log/ceph#
 
Last edited:
Jepp, unfortunately ceph zabbix send isn't quite talkative when it comes to tracing errors. I remember having to set up some settings in Zabbix to make it accept the messages from ceph zabbix send. Maybe thats the part that is missing.

I will look it up, what it was and post it here. But this will take some time, because I'm out of office tomorrow.
 
  • Like
Reactions: sunkroot
Traceback (most recent call last):
File "/usr/share/ceph/mgr/zabbix/module.py", line 303, in send
zabbix.send(identifier, data)
File "/usr/share/ceph/mgr/zabbix/module.py", line 46, in send
stderr))
RuntimeError: /usr/bin/zabbix_sender exited non-zero: zabbix_sender [1925062]: DEBUG: answer [{"response":"success","info":"processed: 0; failed: 44; total: 44; seconds spent: 0.000416"}]

I've attached module.py and renamed it to module.py.txt so that i can upload it.
 

Attachments

  • module.py.txt
    12.6 KB · Views: 13

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!