We're excited to announce the launch of Data Products, a powerful new way to group, monitor, and govern your data assets based on the business value they deliver.

You can now group related assets—such as tables, pipelines, and dashboards—into a single Data Product. This allows you to monitor the health of an entire end-to-end pipeline as one unit, rather than as dozens of separate components.

What you can do with Data Products:

• Monitor End-to-End Health: Get an at-a-glance status (Healthy, High-Risk Incidents, Urgent Incidents) for your most critical data pipelines.
• Establish Clear Ownership: Assign owners to a complete business use case, making it clear who is responsible for the "Marketing Analytics" or "Product Recommendations" data.
• Scale Your Governance: Define default alerting and notification strategies at the product level to ensure consistent standards across all related assets.
• Tie Data to Value: Link technical assets directly to their business purpose with detailed descriptions and SLAs.

How to get started:

• Find the new Data Products page in the main navigation menu to create your first product.
• You will now see a "Data Products" attribute on your asset pages, showing which products they belong to.

Check out our documentation to learn more about how to leverage Data Products to build more trust in your data.

Coming Soon: We're already hard at work on the next wave of enhancements. Look forward to:

1. Data Product Lineage: The ability to visualize dependencies and relationships between your Data Products.
2. Monitoring Strategies: The power to define and apply monitoring rules across all assets within a Data Product to enforce standards at scale.

We're excited to introduce the new Sifflet Dashboard, your central hub for understanding the health and reliability of your data platform at a glance. We've redesigned this page to give you a clear, actionable overview of your data landscape from the moment you log in.

The new dashboard surfaces the most critical information about your data quality and coverage, helping you and your teams build trust in your data.

Here’s what’s new:

• Get a High-Level Overview: Instantly see a summary of your connected data assets, including the number of sources, tables, columns, pipelines, and dashboards available in Sifflet.
• Track Data Health via Actionable Metrics: New charts for Table Uptime and Incidents provide a clear, immediate view of your data's reliability and any ongoing issues. You can quickly assess the status of your freshness, volume, and other monitors, as well as view the daily trend of data incidents.
• Understand Your Coverage: The Monitor Coverage and Monitor Types charts help you see how much of your data landscape is monitored and how your monitors are distributed, making it easy to spot and fill any gaps in your observability strategy.
• Drill Down with Powerful Filters: Use the new powerful filters at the top of the page to focus on the specific data you care about. Filter your entire dashboard view by domain, a combination of tags, or date range (from 7 to 90 days) to quickly investigate issues and get the context you need.

This new dashboard provides a solid foundation for data observability, but we're just getting started. Our next major step is to give you the power to create fully customizable dashboards tailored to your specific needs. Stay tuned!

For a full tour of the new features, check out the documentation.

App version: v508

The 0.4.0 version of the Sifflet CLI will be available on July 7th.

Changes

The sifflet code workspace command has been improved as follows:

• sifflet code workspace apply command:
  • Now shows a plan and asks for user confirmation to apply. Use the --auto-approve flag to directly apply without seeing the plan.
  • Breaking change: Now, untracked monitors are deleted from Sifflet by default. Use the --keep-untracked-resources flag to keep them.
  • Breaking change: The --dry-run flag has been removed. Use the sifflet code workspace plan command instead.
• The sifflet code workspace plan command is now available.
• sifflet code workspace delete command:
  • Now shows a plan and asks for user confirmation to apply. Use the --auto-approve flag to directly apply without seeing the plan.
  • The --keep-resources flag is now available. Use it if you want to delete the workspace but keep the monitors.
• Breaking change: The --verbose flag has been removed. All commands are now verbose by default. Use the --quiet flag to only see errors and a summary.

Migration guide

• If you use sifflet code workspace apply or sifflet code workspace delete in CI/CD pipelines, you should add the --auto-approve flag to skip the confirmation step.
• Replace sifflet code workspace apply --dry-run commands by sifflet code workspace plan
• Remove --verbose flags

We're excited to announce significant improvements to our Databricks integration, bringing deeper context and visibility to your data assets. You can now leverage Databricks tags directly within Sifflet and better trace dependencies with our new field-level lineage for Databricks.

What's New?

• Databricks Tags Now in Sifflet: We now automatically retrieve tags associated with your tables and columns in Databricks and display them in the Sifflet catalog. This enriches your data discovery and governance workflows, allowing you to use your existing Databricks tagging strategy to organize and find assets within Sifflet.

  

• Field-Level Lineage: Our data lineage graphs for Databricks now go deeper. We've added support for field-level (or column-level) lineage, allowing you to see exactly how individual fields are connected across your data pipeline. This granularity is crucial for precise impact analysis and root cause investigation.

  

Coming Soon

This is just the beginning. We're continuing to improve our Databricks integration and will be releasing more enhancements (like Databricks Workflows support) in the near future. Stay tuned!

How to Get Started

To enable these new features, you'll need to ensure your Databricks environment is configured to allow Sifflet access to system tables. For detailed instructions, please see our updated Databricks integration documentation.

App version: v501

We're excited to release the Sifflet MCP (Model Context Protocol) Server, a new way to interact with your Sifflet data observability platform directly from your development environment and other tools!

The Sifflet MCP Server empowers you to perform a variety of data observability operations. For example, you can:

• Understand Downstream Impact: Before changing a dbt model or table, quickly identify all dependent downstream assets and their owners to ensure smooth collaboration.
• Access Up-to-Date Metadata: Get the latest operational details for any table—how it's monitored in Sifflet, any active incidents, frequent users, and more—right when you need it.
• Bootstrap New Asset Monitoring: Easily discover and adapt existing Monitor-as-Code YAML configurations from similar assets to quickly set up robust monitoring for new tables or dbt models.

This server enables more integrated data observability operations, bringing crucial context from Sifflet closer to your workflows.

Explore, contribute, and get started on GitHub. We look forward to your feedback!

We're excited to announce the release of Custom Metadata, a powerful new way to add rich, contextual information to your assets, monitors, and other entities within Sifflet! This feature allows you to define and apply your own key-value pairs, tailoring Sifflet to your specific organizational needs and workflows.

Key New Functionalities:

• Define Custom Metadata Attributes:
  • Create custom metadata fields with a specific Key (e.g., "Data Owner," "Project ID," "Sensitivity Level").
  • Choose from new Value Types for your metadata:
    • User: Assign Sifflet users.
    • String: Add free-text values.
    • Label: Select from a predefined list of options.
  • Specify which Asset Types (Datasets, BI Assets, Pipelines, Declarative Assets, Monitors) each custom metadata attribute can apply to.
  • Configure if an attribute can be used for filtering and if it supports multiple values.
• Apply Metadata to Assets (individually or at scale): Add or update custom metadata values directly in the "Details" section of an asset's "Overview" tab, or at scale by leveraging the Bulk Actions in the Catalog or Monitors page.
• Filter by Custom Metadata: If configured, use your custom metadata attributes to filter assets in the Catalog and Monitors pages for improved discoverability.

Why it Matters:

• Deeper Context: Go beyond simple tags to add specific, meaningful information to your Sifflet entities.
• Improved Organization & Discoverability: Structure and find your assets more effectively using your own defined criteria.
• Enhanced Governance: Track ownership, sensitivity, compliance status, and other critical governance information directly on assets.
• Streamlined Workflows: Easily identify and manage assets related to specific projects, teams, or business units.
• Customized Monitoring at Scale (Coming Soon): Leverage custom metadata to define monitoring strategies at scale based on your own criteria. (e.g., automatically creating Freshness Monitors for all tables that are part of a given project.)

To learn more about setting up and using Custom Metadata, check out our documentation page. We look forward to your feedback as we continue to enhance this feature!

App version: v497

We’re excited to roll out Join support for data monitors!

You can now add joins directly in the monitor setup UI, making it easier than ever to build context-rich data quality checks that span across datasets.

✨ What You Can Do:

• Group by columns in a joined table
  Example: Monitor ORDER volume grouped by CUSTOMER_TYPE after joining with the CUSTOMERS table.

• Filter based on joined table values
  Example: Filter out users where CUSTOMERS.LAST_ORDERED_AT > CURRENT_DATE() - 30.

• Flexible join key selection
  Define join conditions using shared keys (e.g., CUSTOMER_ID) between datasets.

To get started, simply click + Add join in the monitor setup panel, pick your table, set a join key, and build more powerful monitors in minutes!

🛠️ Data Quality as Code: Define Joins Programmatically

Prefer infrastructure as code? You can now define dataset joins in your YAML monitor definitions.

🔧 Example:

datasets:
  - uri: snowflake://sifflet-internal/DEMO.TEST.AGG_CASE_WHEN
joins:
  - dataset:
      uri: snowflake://sifflet-internal/DEMO.TEST.ALLUSERS
    joinCondition:
      kind: Equality
      fieldPairs:
        - leftField:
            dataset:
              uri: snowflake://sifflet-internal/DEMO.TEST.AGG_CASE_WHEN
            name: FIRSTNAME
          rightField:
            dataset:
              uri: snowflake://sifflet-internal/DEMO.TEST.ALLUSERS
            name: FIRSTNAME

You can now define:

• Join conditions
• Join fields
• Datasets and datasources all in YAML — enabling version control and CI/CD for data quality.

🔮 Coming Soon: Custom Aggregations in the Wizard

We're working on something exciting! Soon, you'll be able to define custom aggregations directly in the UI.

Example use cases:

• SUM(conversion_rate * amount_in_original_currency)

Stay tuned for more powerful and flexible monitoring options that don’t require SQL!

For additional details on joining tables in monitors, you can refer to the dedicated documentation page. Want help getting started or converting your current monitors to use joins or YAML? Reach out to your customer engineer — we're here to help!

App version: v494

We’ve evolved the Distribution Change monitor to provide more meaningful insights. Instead of performing a statistical analysis on the entire distribution, it now analyzes the volume percentage for each category within the distribution.

✅ What’s New

Aligned Anomaly Detection: Detection now works just like our other monitors, ensuring consistency across your monitoring setup.

Seamless Migration: We've automatically migrated your existing monitors to the new type. You’ll still be able to view your historical graphs from the previous version.

Mode Mapping:

• Dynamic mode remains unchanged.
• Static mode now corresponds to a relative threshold mode, using your original threshold (in percentage points).

New Control Options:

• Fail if new value appears – alerts you when a previously unseen category is detected.
• Fail if value disappears – alerts you when an existing category is missing from the distribution.

Important note
Group-by monitoring is not supported in this new version. Group-by dimensions previously configured have been migrated to dataset fields analysis. We now analyze the concatenated values of dataset fields and dimensions to maintain coverage.

App version: v493

We've introduced a dedicated Users section within the domain management page, significantly simplifying access permissions management. This enhancement streamlines administration, especially for domains with numerous users.

With this new section, you can now:

• Get a clear overview of all users who have access to the domain, displaying their Username and assigned Role.
• Efficiently add multiple users to the domain in one operation, assigning their specific roles during the addition process.
• Update the domain roles of multiple users simultaneously.
• Quickly remove one or multiple users from the domain when access is no longer needed.

This provides a much-requested, centralized way to manage domain membership and permissions directly where you manage the domain itself. You can find this new section within the settings of any domain. (Learn more)

App version: v480

We've enhanced the "Test Connection" process for BigQuery and Snowflake. It now thoroughly checks permissions and provides precise error feedback, telling you exactly what's missing if a permission check fails. This improves initial setup and debugging of permission issues during source refreshes.