Webhook Notifications, empty fields, and PagerDuty

[AwakeBread]

I'm trying to set up PagerDuty notifications in Proxmox VE 8 and I want something like this:

{
  "payload": {
    "summary": "{{ escape title }}",
    "severity": "critical",
    "source": "{{ fields.hostname }}",
    "custom_details": {
      "message": "{{ escape message }}",
      "hostname": "{{ fields.hostname }}",
      "original_severity": "{{ severity }}"
    }
  },
  "client": "Proxmox",
  "routing_key": "{{ secrets.pd }}",
  "event_action": "trigger"
}

The webhook fails with the confusing error `Could not test target: https://events.pagerduty.com/v2/enqueue: status code 400 (500)`

After way too much troubleshooting I discovered that test notifications apparently don't populate the "fields" and PagerDuty requires a "source".

Is there any way to set this up with default values in case one is missing? Is there any way to map the Proxmox severity values to the PagerDuty ones? Is there anyway to test your template other than throwing it at a web endpoint and seeing what happens?

 

[Lukas Wagner]

Hello AwakeBread,

Is there any way to set this up with default values in case one is missing?

You can use Handlebar's if statements [1] to fall back to a default, like so:

{
  "hostname": "{{#if fields.hostname }}{{ fields.hostname }}{{else}}localhost{{/if}}"
}

Is there any way to map the Proxmox severity values to the PagerDuty ones?

Not at the moment, no. We would need a generic mapping function for this. Maybe we could provide some custom Handlebars helper function that takes pairs of arguments, each argument pair consisting of the original value and the value to replace it with.

Is there anyway to test your template other than throwing it at a web endpoint and seeing what happens?

Also no, not at the moment. I've been toying with the idea of adding a 'notification simulator' mode, where one can trigger different kinds of notifications (instead of just the regular "Test" notification), so that one can test their matcher configuration. Would also make sense to have some verbose output for all rendered templates, such as the notification text itself, and also, as in your case, the webhook body. This is a pretty big task, I cannot provide an ETA for this at the moment.
Something that could be added much sooner would be some kind of debug mode, one where we just print the rendered template to the logs.

Hope this helps!

[1] https://handlebarsjs.com/guide/builtin-helpers.html#if

 

[Lukas Wagner]

After way too much troubleshooting I discovered that test notifications apparently don't populate the "fields" and PagerDuty requires a "source".

And yes, the test notification does not provide any fields at the moment. Could make sense to add the commonly used ones (e.g. hostname, fqdn), even if we don't use them in the default notification template.