The incidents list page got revamped to improve the overall incident search user experience by making it consistent with the rest of the Sifflet application. By default, only Open and In progress incidents now show on the list, making it easier for you to focus on data quality issues that require immediate attention.
There are some monitors for which you don't necessarily want to create a Data Incident. You can now control Sifflet's automatic creation of an incident via the bulk edit from the Monitor list page.
This feature works also with dbt monitors.
In addition to the bulk edition of monitors, we've also made this toggle available at the dbt datasource level.
🛠 Fixes
Fixed Technical Error in some instances of Duplicate and Null monitors with Group By
Fixed Field Duplicate and Completeness monitors reporting incorrect results for empty datasets
Assigning the appropriate severity to a monitor is instrumental to ensure data quality issues are tackled with the right level of urgency by your teams. You can now bulk edit your monitors' severity, making it easier for you to update your monitor suite to ensure it is set to the proper criticality level.
Simplified Compliance With 20+ New Classification Tags
Sifflet AI Assistant recognizes 20+ new classifiers such as Email Address, Credit Card Number, IP Address or Login Details. Sifflet AI Assistant detects columns to which these classifiers apply and suggests them as classification tags, helping you categorise your data and consequently simplify data governance and compliance with regulation.
Sifflet Insights Now Features Ownerships for Simpler Collaboration
Sifflet Insights (Sifflet browser extension) now surfaces information about assets' owners. This means data stakeholders can now know who to contact in case of issue or question on the dashboard they are looking at without even leaving their BI tool.
You can now enrich your Data Catalog in one-click thanks to Sifflet AI Assistant new capability. Clicking the Generate metadata button on your data assets now indeed generates:
Columns classification suggestions to help teams categorize Tables & Views assets' columns in a more accurate and comprehensive manner.
Descriptions recommendations to enrich data assets documentation and ensure the content of datasets' columns is clear to all data stakeholders.
You can now select multiple tables when creating a Custom SQL monitor, this will allow the monitor to be linked to multiple tables at once but what it also truly unlocks is the power of AI to generate even more advanced Monitors, joining tables automatically and on demand !
Sifflet's AI Assistant will attempt to join tables together with the context of the tables' schemas and column descriptions! (if you identify your column as foreign key in the description that will be taken into account! )
🛠 Fixes
[Data Catalog] Fixed a bug that was preventing some search parameters (search terms, search on fields selection, page, and number of items per page) from being maintained during a Data Catalog navigation.
Sifflet has in the past primarily been doing Freshness based on the distribution of the data, checking that data points with new dates were added. However this does not cover all freshness usecases, such as reloading a dimension table without a date field. In those cases what I want to check if if the data reload occurred within the usual time range !
New Template: Freshness (Update Time Gap)
This template can be applied to any table and will use the Table Metadata available to build up a history of typical time gaps between updates.
Due to the lack of history available, this monitor needs training time to be effective so you will need to wait a few data update cycles for a nice looking graph !
Supported Databases:
Snowflake
Bigquery
Databricks
Oracle
Mysql
🛠 Fixes
Fixed a bug that caused recurring data source update failures in some large environments.
Release: New monitors wizard, Rolling aggregation for monitors with dynamic thresholds
✨ Feature Highlights
New monitors wizard
We have improved the monitor's creation experience on the UI by introducing a redesigned wizard. This new user experience streamlines navigation within the monitor's templates, providing users with more precise guidance on setting parameters tailored to their specific use cases. We will continue improving this experience with more to come in the coming releases.
Rolling aggregation for monitors with dynamic thresholds
Introducing a new addition to our monitoring capabilities, detect anomalies using rolling aggregation alongside dynamic thresholds. Every monitor run will aggregate data based on user-defined rolling time intervals. For instance, users can set up monitors to run daily and detect anomalies in the total number of orders over the past seven days using dynamic thresholds. This feature is compatible with the following monitor templates:
Release: Improved Dataset Selector for Monitoring.
✨ Feature Highlights
Search Datasource Names, Databases and Schema Names in Monitoring Asset Selector
When looking for the right asset to monitor you may find that you have multiple copies of the same name! If you know the datasource name, schema or database you are now able to add those to your search and find your asset faster
