Offset Standardisation

by Martin Zerbib

Previously in Sifflet Offsets could be a bit confusing. No offset for an hourly monitor would monitor the last complete hour, and No Offset for a daily monitor would monitor the current day.

We're standardising these options for all monitor granularities

Now:

  • Default Offset: 1 - A default offset of 1 will query the last complete timeslot
  • Zero Offsets: Setting offsets to 0 will query the current timeslot , the current day, the current hour. This is good when your days data should be complete by a certain time or if you want to compare a value at a specific time of day!

There is no action to change anything on your side!

Data Quality as Code

While your previous configs will still work we're introducing a new parameter to streamline this. Offsets previously needed you to specify a granularity, leading to potential confusion. This new offsetPeriodsparameter will automatically account for your time period granularity.

  timeWindow:
    field: auto
    firstRun: P365D
    offsetPeriods: 2

We suggest using this version moving forward !

When a monitor is failing and a resolution is planned but will take time, you really don't want to keep receiving alerts telling you something is wrong !

We've introduced a new Muting capability to stop alerts. Simply find the little bell Icon in the monitor action bar to mute monitors !

What's next? Dynamic muting options such as "mute until passing"

Previously, when forgetting your password, you had to reach out to a user with an admin system role to have them reset your password so you could access your account again.

We're introducing a streamlined and secure "Forgot password?" workflow to make account recovery faster and easier. You can now reset your password in just a few clicks directly from the login page, with email-based verification ensuring a safe and user-friendly process.

Note: The "Reset password" workflow remains available, as it can be handy for self-hosted Sifflet deployments that don't include a SMTP server.

Read more about the "Forgot password?" workflow

App version: v446

Slack integration was improved to support automatic messages updates. Whenever a monitor returns to an OK state, the original Slack alert message now updates in real-time to reflect the resolution, keeping everything clear and up to date.

This enhancement ensures your team instantly sees when issues are resolved, reduces channel clutter, and keeps everyone aligned without extra effort. Stay focused and responsive with more efficient incident communication in Slack!

Read more about Slack integration

App version: v443

Data lineage is a crucial tool for understanding how data flows through your systems. However, large and intricate pipelines can quickly become overwhelming. With the new lineage node retraction feature, you can hide specific nodes from your data lineage view. This makes it easy to:

  • Focus on What Matters: Hide less critical nodes to simplify your view.
  • Speed Up Troubleshooting: Quickly identify root causes and dependencies without visual noise.

This enhancement gives you the flexibility to declutter complex lineage graphs by collapsing nodes that are not immediately relevant to your analysis, enabling clear navigation of your data ecosystems.

Read more about lineage

App version: v435

Webhooks were enriched to send a new type of events. Sifflet webhooks now send real-time notifications for:

✅ Monitor Success Runs: Get notified when a monitor runs successfully, providing confidence that your data quality test went smoothly and that the latest verified data is reliable

❌ Monitor Failing Runs (Existing): Continue receiving events when a monitor detects an issue, ensuring you never miss a potential problem.

These new events allow you to push more comprehensive Sifflet monitor insights to your notification, ticketing, or data catalog tools as well as build even more sophisticated programmatic workflows.

Read more about the webhook integration

App version: v434

The Filter conditions method now supports the OR operator between conditions when creating a domain. This enhances flexibility by allowing you to define domains containing assets that either come from specific sources or are tagged with a specific tag or label.

Read more about domains

App version: v433

There are two scenarios for monitors going from Failing to Passing

  • Monitors that scan the entire table: When the last run of the monitor succeeds the monitor will change status to Passing because the entire table has been scanned and the anomaly is not present anymore.
    i.e. A monitor detected 50% null values on a column, and the next run detects 0% null values, the problem has been fixed.
  • Incremental Monitors: Incremental monitors require human interaction to go from failing to passing, by qualifying all the anomalies. Why is the behaviour different? because a successful run on an incremental monitor does not mean the issue has been resolved. For example if there were null values in yesterday's data, but there are none in today's data, there is nothing that suggests yesterday's data has been fixed.

We recently introduced Dynamic Monitor Statuses , allowing monitors to change their status based on qualifications made by the user.

A new flow will now ensure that qualifying a monitor to make it passing will now close the incident associated automatically.

Step 1: Qualify a Monitor as Passing

Step 1: Qualify a Monitor as Passing

When qualifying a monitor as passing, either via manually qualifying points or by using the global monitor Qualify as passing option, this will now propagate to the incident.

Note: An incident with multiple monitors will only close automatically if all the monitors are passing!

📘

Coming Soon

Closing an incident will currently not propagate to monitors, this is coming soon !