Custom SQL monitors without having to write any code
Custom SQL monitors are powerful, however they rely on strong SQL skills... No More !
Sifflet's AI Assistant can now generate SQL for you! And that's not all, Sifflet leverages the power of all the documented context of your catalog to better understand your prompts!
Sifflet's AI Assistant understands the context of your data
Create powerful monitors in a few seconds. Sifflet identifies the best fields based on catalog documentation. Order_Cost = order preparation
Apply complex Business Logic checks in seconds
Available For Custom SQL monitors. Coming soon to Custom Smart Metrics !
Fix SQL with AI for Custom SQL Monitors
Sometimes it's hard to write SQL correctly, if the Test Run fails Sifflet will now display the option to use AI to attempt to generate correct SQL!
You can now sort business terms by name or by number of assets & monitors using them. This makes it simpler for your teams to navigate the Business Glossary and find terms that matter the most to them.
Release: Preview Sample Data of All Table Columns in One-Click, Email Integration UI Revamp
✨ Feature Highlights
Preview Sample Data of All Table Columns in One-Click
You can now preview a sample of all your table columns in one-click, making it even simpler to get an overview of what your table data actually looks like.
In addition to recent changes ensuring Sifflet users' email addresses are automatically configured (See v216 changelog), the email integration just got entirely revamped in order to:
Give it a fresh look and make it easier to differentiate users' email addresses from custom ones
Simplify the configuration of multiple custom email addresses in one go
Allow you to delete custom configured email addresses from the integration and monitors
A long awaited feature ! You can now add qualification comments to graph qualifications in Sifflet.
Qualification comments show in the graph, alongside the user who qualified the point and the date of qualification. Look out for little dots on qualified points, they indicate a comment is available!
Release: Sifflet Insights Chrome Extension for Power BI
✨ Feature Highlights
Sifflet Insights Chrome Extension for Power BI
Sifflet Insights now supports Power BI, therefore going one step further in helping teams feel confident about the data in their BI dashboards.
By surfacing up-to-date data catalog and data quality issues information directly in your BI tool, Sifflet Insights indeed makes it easy for dashboards users to understand what the visualised dashboard is about and whether or not its data is reliable.
Release: Model training improvements, lookback period improvements, improved SSO to support BoxyHQ
✨ Feature Highlights
Model Training Improvements
Optimal hyper parameters for machine learning models are now computed and applied on initial run. Ensuring the first run of the monitor runs optimally. Hyper parameters for each model are also periodically improved and recalculated by sifflet.
Lookback Period Improvements
Monitors now alert on the Lookback Period. The lookback period is used to add a past buffer to monitor runs to account for pipelines who update or backfill past data points. This was previously used only to update graphs and improved the model for better predictions.
What Changed: If a data point in the lookback window was within the expected range at the time of the initial run but has since then been changed the Sifflet monitor will treat that change as a new anomaly
Additionally if a data point was an anomaly when it was first generated but a subsequent run identifies that the point is now in the correct range. It will automatically be marked as Fixed
A Point is an anomaly
The next day's run has a lookback period of more than 1, it checks yesterday's point and identifies that the value is now correct.
Release: Automatic configuration of users' email addresses
✨ Feature Highlights
Automatic configuration of users' email addresses
Sifflet users' email addresses are now automatically provisioned to your mail integration. Your teams consequently no longer have to first go through the step of configuring their personal email addresses at the mail integration level before using them in monitors, making the first step to monitor creation swifter.
Sensitivity for Anomaly detection has always been a complex issue. To reduce noise, it is ideal to find the best sensitivity value that fits the most valid points and still detects anomalies in the data when they do occur. Sifflet has now switched from having set sensitivity thresholds to allowing for even more granular control of anomaly detection.
Mapped values to previous setup: Low -> 25, Medium -> 50, High -> 85
ML Monitoring - Sensitivity recommendations
Along with the change, Sifflet's AI assistant can now deliver more specific and efficient Sensitivity suggestions, aiming to fit as many valid points as possible (including those marked as false positives !) and still detect anomalies for true errors!
Monitors - Incident Creation Toggle
There are some monitors for which you don't necessarily want to create a Data Incident, so we've added a toggle that let's you turn off SIfflet's automatic creation of an incident when a monitor fails. This is great for monitors you want to review periodically without initiating any resolution processes for!
We use this feature on some of our internal sifflet instance where we detect anomalous patterns in Sifflet usage. If a Customer is creating more monitors than usual we want to proactively reach out to them to help but we don't necessarily need an incident for that! 💡
Monitors as code:
incident: # (REQUIRED)
severity: # (REQUIRED) Severity of the incident
"Low" | "Moderate" | "High" | "Critical"
message: String # (optional - default null) Custom message to add to the incident and notifications
createOnFailure: Boolean
Monitors - Search Improvements
We've improved the monitor search page to further improve the way you the monitors you want.
Improved freetext search patterns
Count previews in filters to identify quickly how many monitors belong to each filter option
User management API endpoints were updated to allow you to programmatically associate roles to the domains your Sifflet users have access to. This simplifies access controls management and ensures only the right people access the appropriate data inside of Sifflet.
A dbt Cloud outage last week temporarily broke our ingestion of dbt test runs for dbt Cloud integrations. Only customers that ran tests during the outage were affected. We have introduced extra error handling to handle the unforeseen "malformed" run data reflecting the impacted test runs.
Data Catalog - Search Failures
Fixed a bug that was causing searches on the Data Catalog to fail when too many filters were selected.
When changing parameters for a monitor with a historical time window, Sifflet will sometimes refetch the historical data to ensure the monitor has an accurate graph representation of the data being monitored.
For example, when adding a WHERE clause to a monitor, Sifflet refetches the history to ensure all past data points reflect the filtered data rather than the data without the WHERE clause. This also ensures the model is trained on accurate data.
Optimisations to the historical refetches have been made to retrigger only when needed. A monitor will now avoid a refresh when sensitivity is changed or when the time window period is reduced and will trigger only for cases such as increasing the time window period, filtering the data on a new condition or grouping the data by a subcategory.
