Incident-Centric Notifications & New Notification Templates

Notifications have been rebuilt around incidents. Sifflet now tells you what happened to an incident, instead of sending one alert for every failing monitor, and every notification template has been redesigned.

Incident-centric notifications

When a monitor fails and creates or joins an incident, your destinations are now notified about the incident:

• Incident created: a new incident has been opened, with the monitor that opened it.
• Incident updated: another monitor was added to an incident that already existed.
• Incident closed: the incident is closed. The notification is sent to every channel the incident has notified so far, not only the one that opened it.
• Monitor recovered: a monitor in the incident is passing again.
• Assignee added: each user newly assigned to an incident receives an email, including users reached through a team assignment.

For an incident, a notification is only sent for the first failed run of a monitor. Subsequent failed runs of the same monitor do not trigger additional notifications until the incident is closed, which cuts the alert noise on recurring failures.

Monitors that do not create an incident keep sending monitor-centric notifications for each failure, needs-attention run and recovery.

Redesigned templates for Slack, Email, Microsoft Teams and Google Chat

All notification templates have been revamped, including the monitor-centric notifications for monitors that do not create an incident. The same notification now looks different on each channel but says the same thing.

• Slack and Email: notifications about the same incident are grouped into threads. On Slack, the original incident message is also updated in place when the incident is closed. Threads and in-place updates apply to incident-centric notifications only.
• Google Chat: now receives incident notifications and can be selected as a destination in notification rules.
• Email: monitor recoveries are not sent by email. Use Slack, Microsoft Teams, Google Chat or a webhook to be notified when a monitor is passing again.
• Webhooks are not templated and keep their JSON payload. They are sent for every failed run of a monitor, incident-centric or not, and also when the monitor is muted. See the Webhook Notifications updates release note for the new INCIDENT_CLOSED event and payload fields.

Why this matters: you get one clear conversation per incident instead of a stream of repeated alerts, and each channel carries the right level of context for the people who read it.

Learn more

The new Notifications section of the documentation lists every notification and the channels that receive it: Notifications overview and Notification catalog.

Questions or feedback? Reach out to your Sifflet contact.