Data catalog asset pages now feature much more detailed monitoring insights about your tables and their associated columns. These insights make it easy for your teams to get a grasp of your assets' health status in one glance: is the number of rows coming in for this table expected? Is the data fresh? Has there been any changes in the schema that could impact downstream assets? Are there any format weirdnesses in columns? Etc.
Having clear asset health statuses along with details about key monitoring dimensions is critical to ensure data trust among your team and can also be a great starting point to an investigation when noticing unexpected behaviors.
Sifflet now fully supports Dynamic Tables within our Snowflake integration. With this enhancement, Dynamic Tables will:
Be included in the Sifflet catalog (and can be monitored)
Appear in the lineage graph for greater visibility
To enable Dynamic Tables in your Snowflake sources, ensure the following permissions are granted to the role used by Sifflet:
GRANT SELECT ON ALL DYNAMIC TABLES IN SCHEMA <DATABASE_NAME>.<SCHEMA_NAME> TO ROLE <SIFFLET_ROLE>;
GRANT SELECT ON FUTURE DYNAMIC TABLES IN SCHEMA <DATABASE_NAME>.<SCHEMA_NAME> TO ROLE <SIFFLET_ROLE>;
GRANT VIEW LINEAGE ON ACCOUNT TO ROLE <SIFFLET_ROLE>;
Catalog your MicroStrategy dashboards and reports and enrich them with Sifflet metadata (owners, descriptions, tags, etc.).
Access the upstream lineage of your MicroStrategy assets to evaluate the impact of data incidents and determine the quality of the data/metrics in the BI layer.
Create monitors upstream of MicroStrategy assets to ensure that data incidents are detected before the data reaches your dashboards.
Have real-time access to the impact of data incidents on a given dashboard or report without leaving MicroStrategy, thanks to the Sifflet Insights Chrome Extension.
Lineage graph with MicroStrategy assets
Ready to get started? Visit our integration setup guide or contact your customer success manager for more details!
We’ve enhanced our platform’s authentication experience to allow users to log in to Sifflet using either Single Sign-On (SSO) or a traditional username/password combination.
This optional feature provides greater flexibility for organizations with diverse security and user management needs. Teams can leverage SSO for a centralized authentication through their preferred identity provider (IdP) while still maintaining the option for direct login when needed. This ensures accessibility for all users, including contractors or partners who might need to access Sifflet data observability insights without being able to sign in through the organization's IdP. It also ensures you can access Sifflet at all times, including in the event of an IdP outage.
We’ve completely reimagined how Sifflet integrates with dbt to make your workflows smoother and more powerful. This revamp introduces exciting new features that seamlessly combine dbt models with datasets and enrich your lineage and catalog experience with dbt metadata.
dbt Models + Datasets: A Unified Asset Experience
Previously, dbt models and the datasets they generated existed as separate entities in Sifflet, with distinct catalog entries and asset pages. With this release, we’ve combined them into a single asset and brought new dbt metadata to Sifflet. Here’s what this means for you:
In the Catalog: Enhanced Metadata for dbt-Generated Datasets
Datasets created by dbt now include key dbt metadata:
Last Execution Timestamp: See exactly when the corresponding dbt model was last run.
Last Execution Status: Quickly identify if the latest execution was successful, failed, or skipped.
Dataset catalog entry with dbt metadata
On the Asset Page: Introducing the New dbt Tab
The dataset’s asset page now features a dedicated dbt tab, consolidating previously scattered dbt information in one place. This is just the beginning—soon, this tab will include even more insights like the model's group, its access modifier, and custom dbt metadata defined using the meta field.
The brand-new dbt tab
Streamlined Lineage Graph: Fewer Nodes, More Insights
The Sifflet lineage graph is now cleaner and more intuitive. dbt models are no longer displayed as separate nodes. Instead, their metadata is integrated into the dataset node, reducing clutter and redundancies. The result? A more streamlined view with richer, consolidated information.
The new lineage graph with dbt metadata
Looking Ahead: More dbt Features on the Horizon
This is just the first phase of our dbt integration revamp. Here’s a sneak peek at what’s coming next:
Cost & Performance Monitoring: Gain insights into the resource usage and efficiency of dbt runs.
Leveraging Custom Metadata: Use dbt’s custom metadata directly in Sifflet for advanced configurations.
dbt-based Monitor Setup: Define and configure Sifflet monitors directly within your dbt YAML files.
We’re excited about this leap forward and hope you are too. Want to see it in action? Reach out to our team to learn more!
You can now customize the system and domain permissions you want to grant users created through Just-In-Time (JIT) user provisioning. This addition allows for operational efficiency while minimizing risks of over-privileged accounts.
Monitors in Sifflet can be created in multiple ways, whether they come from DBT test runs, are created via the interface, or are deployed within a code-based workspace. New filters in the Monitor page allow for filtering based on these creation methods!
Custom Thresholds for SQL and Conditional Monitors
We recently added the ability to apply custom threshold settings to many monitor templates in SIfflet, we're bringing this functionality to SQL and conditional Monitors.
Rather than alert on any matching result you can now alert:
Static: If more than X results match the condition
Dynamic: If the number of results diverges from the usual trend
Relative: If the number of results increases or drops by a certain value/percent
You can now use the Webhooks integration to trigger events to your endpoints in case Sifflet detects a data quality issue. This makes it possible for your teams to integrate Sifflet with virtually any collaboration tools they might be using (e.g. ServiceNow, PagerDuty, Google Chat, etc.). Incorporating data quality issues into your teams' existing tools and workflows is key to ensure streamlined and efficient operations.
Webhooks can also be leveraged for issue remediation purposes. By triggering custom scripts upon data quality issues detection, webhooks indeed allow you to decrease time to resolution by automating predefined actions (e.g. restart processes, reprocess data, or reroute workflows based on the nature of the issue).
