Databricks Workflows Integration

You can now connect Sifflet directly to Databricks Workflows to gain complete, end-to-end observability of your data pipelines and the assets they generate. This new integration allows you to monitor your data orchestration alongside your data quality in a single platform.

Key features include:

• Automated Job Discovery: Sifflet now automatically discovers your Databricks jobs and populates them as assets in the Sifflet Data Catalog. Each job asset page centralizes key metadata, including run status, tags, and ownership.

  

• Enhanced Data Lineage: See exactly which Databricks Workflows jobs are creating or updating your tables with an enriched, end-to-end lineage view. This helps you understand the impact of pipeline changes and troubleshoot issues faster.

  

• Coming Soon: AI-Powered Root Cause Analysis: By understanding the relationship between your jobs and data, Sifflet's AI assistant, Sage, will be able to identify failing Databricks jobs as the root cause of data incidents.

To get started, please refer to our new Databricks Workflows documentation.

App version: v564