Conditional Monitors

Overview

The Conditional Monitor is Sifflet's most flexible no-code rule builder, available in the "Custom" monitor category.

It's designed to create powerful data quality rules with multiple conditions (like IF-THEN logic) using a simple, visual interface—no SQL required. The monitor alerts you if any rows in your dataset match the conditions you define.

Following a recent upgrade, this monitor now supports Sifflet's full range of monitoring capabilities, including multi-dataset joins, incremental scans,group_by, and value mode.

ℹ️

Supported Databases

Sifflet supports Conditional Monitors on all major data warehouses, including:

• BigQuery
• Snowflake
• Databricks
• Redshift

This list is always growing. Please check your Sifflet instance for the most up-to-date list of supported data sources.

When to Use a Conditional Monitor

Use the Conditional Monitor when you need to:

• Validate logical integrity: Check if updated_at is always greater than created_at.
• Enforce business rules: Ensure that if product_category = 'Clothing', the tax_rate is 0.08.
• Perform cross-dataset validation: Join your Orders and Payments tables to find orders marked as 'PAID' that have no corresponding successful payment record.
• Check for invalid combinations: Alert if country = 'USA' and zip_code is not 5 digits

How to Use

1. Datasets & Joins

You can build a rule on a single dataset or join multiple datasets to create complex cross-system validations.

• Select your datasets: Start by choosing your primary dataset.
• Add joins (Optional): You can add one or more datasets to join with. This uses Sifflet's standard "join-everywhere" engine.
• Define Join Type: Select the type of join you need:
  • Left Join: Keeps all rows from the left dataset.
  • Right Join: Keeps all rows from the right dataset.
  • Inner Join: Keeps only rows that match in all joined datasets.
  • Outer Join: Keeps all rows from all datasets, whether they match or not.
• Select Join Keys: Choose the fields from each dataset to join on (e.g., Orders.user_id = Users.id).

2. Define Conditions

This is the core of the monitor. You are building a set of rules to find "bad" data. By default, the monitor will alert if any rows match these conditions. You can, however, leverage the powerful Threshold Settings to tweak this.

You can create conditions by comparing fields to other fields or to static values.

Column Conditions

Compare a field to another field. This is useful for logical checks.

• Example: updated_at (Column) Less Than created_at (Column)

Value (Constant) Conditions

Compare a field to a fixed value. This is used to check for specific states, values, or nulls.

• Example 1: status (Column) Equals ERROR (Value)
• Example 2: user_email (Column) Is Not Null (Value)

Available operators include:

• Equals / Not Equals
• Greater / Greater or Equals
• Less / Less or Equals
• Contains / Not Contains
• Is Null / Is Not Null

Multiple Conditions and Groups

You can add unlimited conditions and combine them with AND / OR logic. You can also group conditions together to build sophisticated logic.

• Example: ( (status = 'Shipped' AND shipped_at IS NULL) OR (status = 'Delivered' AND received_at IS NULL) )

Unleash Full Monitoring Power

Conditional Monitors now support the same advanced features as all other Sifflet monitors.

Filtering (WHERE Clause)

Use the standard Where filter to narrow the scope of your monitor before your conditions are applied. This is ideal for focusing your rule on a specific segment (e.g., country = 'FR' or event_type = 'purchase').

Segmentation (GROUP BY)

You can now use the Group By feature. Instead of one global alert, the monitor will create separate issues for each data segment, allowing you to pinpoint problems (e.g., group by: [store_id]).

Threshold Settings (Count vs. Percentage)

You can choose how the monitor reports issues:

• Row Count (Default): The monitor alerts if the number of rows matching your conditions passes a threshold (e.g., "Alert if count > 0").
• Percentage: The monitor alerts if the percentage of rows matching your conditions passes a threshold (e.g., "Alert if count > 5%").

You can then choose the threshold mode (dynamic, static, or relative) and the specific anomaly conditions.

Time & Incremental Scans

Conditional Monitors now use Sifflet's standard Time Window configuration. This allows you to:

• Run Incremental Scans: Configure the monitor to scan only new or updated data, which is highly efficient for large tables.
• Define Time Windows: Easily focus your monitor on "data from the last 14 days" or "data from the last 2 hours" using the standard time parameters.

This new integration replaces the legacy TIME PERIOD operator, though that operator remains available for specific, non-standard time-slicing needs.