Only notify after X amount of failures?

[artye]

We run Proxmox clusters that are used for live event production, so they get brought up and down fairly often as they are loaded into a new venue for a show or get sent back to our shop for maintenance. The VMs have replication jobs running every 5 minutes so HA can pick up should a node crash. However, if the nodes are booting or shutting down, we frequently get a flood of "replication failed" email notifications as one is booted but the others are still booting.

Is there a way to only notify after X amount of failures? Or maybe a way to detect that a shutdown was intentional and not to notify as an error?

 

[bbgeek17]

Hi @artye , welcome to the forum.

This functionality is best implemented on the receiver side. If you are sending it to an alert system, the smart-ones usually have a rate limit.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[artye]

So far we're just using the built-in email notification system.

 

[bbgeek17]

So far we're just using the built-in email notification system.

I understand. You should direct the email to mailbox/system that can process the emails, batch similar events and escalate based on defined counters.
This is not something that a Hypervisor would excel at. There are companies that are dedicated to building and maintaining alert processing flows.

To answer your questions - no, there is no way to do this in PVE.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox