Being alerted by monitors allows you to be proactive about issues detected by Sifflet and to react before the issue impacts your teams and any decision they might be about to make. It also allows you to be alerted in case of a monitor misconfiguration to ensure that it doesn't go unnoticed and that your monitoring coverage remains intact.

📘

Before setting notifications, make sure to setup alerting channels

Before any Notifications may be configured, Collaboration Tools need to be connected.

How to

Setup Notifications

Overview

Notifications are messages sent through chosen communication channels when a Monitor doesn't succeed, triggering an Incident creation. A Notification's main goal are making sure the information reaches the right stakeholders as soon as possible and facilitating debugging.

Collaboration Tools

In order to be able to configure Notifications, remember to setup Collaboration Tools beforehand.

Monitor-level setup

Notifications configuration is available as the last step of a manual Monitor configuration process and it can be revisited at any time. It's possible to choose multiple Alert Channels for the same Monitor. Furthermore, it's possible to customize the Alert Message, so that it'd provide the right information to the informed stakeholders, facilitating troubleshooting or providing guidance on what should be done in case of that particular Monitor failing.

Read Notifications

Notifications are triggered if a Monitor Run doesn't succeed and instead results in one of the following Monitor Run Statuses: Failure, Requires attention, or Technical error.

Independently from a Collaboration Tools, a Notification contains the same level of information. Below is an example of a Slack Notification.

Title

The title of the Notification follows a syntax below:

[Alert emoji ] [Monitor Name] [on] [Asset Name] [Monitor Run Status]

Message

The message configured in the Monitor Configuration Step. This message is "fixed" - its content is defined by the user and doesn't change depending on e.g. the Run Status.

Reason

The reason that caused the monitor to fail.

Timestamp

The time of the failed Monitor Run.

Optionally: Window of scanned data

This section reflects the Time Window Monitor setting. It shows up only if a Monitor has that parameter configured.

Links

See Failing Monitor

It redirects to Monitor Page and is normally a good starting point for troubleshooting.

See Incident

Redirects to the Incident Page to get more insights into the impact of your asset on the rest of the business.

Notifications Best Practices

Pick the right alerting channels

• Make sure to pick the appropriate channels depending on the severity of your monitor: if your teams are usually more reactive on Slack or MS Teams, you probably want to have your monitors with the highest severity notify over there and keep emails for the least critical monitors.
• Be mindful of single points of failure: notifying a single person over email is nice as it gives clear ownership, but you should also account for cases where the usual owner of a monitor/data asset might be out: somebody else on the team needs to be able to see the alert and react accordingly if needed.
• Notifications are not only useful to let data owners know that something needs to be fixed, they can also allow you to give visibility to data consumers about the health of upstream pipelines.

Make the alert actionable

• Ensure that your notification message includes the necessary context to facilitate troubleshooting. You can therefore include the information below in your notifications:
  • Debugging guidelines
  • Internal documentation links
  • Etc.

As Code

Parameters

In notifications

Kind: The kind of notification hook you're setting up in the monitor:

name: The name of the collaboration channel or the email .

notifications:
- kind: Slack
  name: auto-demo
- kind: Slack
  name: auto-demo-slack
- kind: Email
  name: [email protected]
- kind: MicrosoftTeams
  name: General

incident.message: A custom message that will be added to alerts

incident:
  severity: Low
  message: If this monitor fails I'll eat my shoe
  createOnFailure: false