There are many scenarios where certain days are going to be anomalies no matter what, and we want to avoid sifting through a sea of alerts to classify them as normal ! We've recently introduced a new functionality to exclude specific days from alerting and anomaly detection.
When creating monitors with a time window setting you can now specify calendars for which you want to exclude dates from anomaly detection. The Calendar list comes preloaded with public holidays calendars for multiple countries as well as other specific calendars to ignore Sundays or Weekends, while still ensuring the graph stays complete for these dates !
When a calendar is active on the monitor points that fall out of the range will be displayed as grey and won't trigger an alert, they will also indicate that the date is being excluded because of a specific calendar !
Custom Calendars
Sometimes standard calendars don't suit everyone's needs! You can also create your own custom calendars via the API
Programmatically Create and Maintain Sifflet Sources
You can now programmatically create and maintain your Sifflet sources thanks to our new sets of API endpoints for credentials and sources. These new endpoints make it easier for your teams to ensure the appropriate data observability coverage by simplifying the roll out of Sifflet on your data stack.
Support for empty and whitespace strings in Null monitors
For textual fields, you can now select what values to check in Null monitors. This capability is provided both for monitors set up in the UI or in Monitors as Code (via the new nullValues parameter).
Sifflet now supports the usage of dynamic parameters in Power BI source expressions, allowing the generation of accurate lineage even for dynamic data sources.
Amazon QuickSight and Fivetran URIs
Sifflet now supports URIs for Amazon QuickSight and Fivetran. You can now consequently tie declared assets to existing Amazon QuickSight and Fivetran assets, allowing you to go one step further in comprehensive data observability with end-to-end lineage and data cataloging.
Managing Incidents is a big focus area for Sifflet at the moment as we know a lot of our customers want to manage incidents at scale! The new and improved Incident Search page allows you to now batch edit the Severity, Status and Assignments of incidents.
🔥 Improved Monitors as Code: For Loops and Templates
Many improvements to Monitors as Code today !
Alternative to Monitor UUIDs when defining monitors: friendlyId
Before:
For each monitor a UUID needed to be specifiedid: 7edf1177-1a3c-4d71-b85f-e38b773735b4 which was often difficult since a UUID had to be generated outside of the yaml.
friendlyId alternative
friendlyId is an alternate ID that only needs to be unique per dataset, this means a dataset cannot have two monitors with the same friendlyId.
kind: Monitor
version: 1
friendlyId: myMonitor
name: My Monitor
extends:
- missionCriticalTags
...
---
kind: Monitor
version: 1
friendlyId: myOtherMonitor
name: My Other Monitor
extends:
- Unimportant
...
For Loops
Before Once again before each monitor had to be defined individually! However there are scenarios where I want to deploy the same monitor on multiple tables !
Now For loops allow you to define monitors on multiple tables easily !
Access Reference Assets Details for Improved Data Observability Coverage
You can now click reference assets on the lineage to view details such as the asset name. This makes it simpler to identify where these assets are located (i.e. in which database, schema, etc.) and consequently create the corresponding Sifflet data sources, enhance your overall data observability coverage.
We've introduced the capability to generate Monitor names and descriptions with AI. This will ensure you have highly descriptive descriptions and names for your monitors with low effort!
How? Simply select the AI icon in the text fields to write custom monitors. You will need to regenerate when monitors are now.
Tip: Not satisfied with the description, click it again and it will change!
Note: This generation is currently not aware of other descriptions and monitor names you have set on other monitors. Future versions aim to leverage these and make it even easier to standardise naming practices.
Need Help? Send Us a Message From Within the Sifflet App!
You can now reach out to the Sifflet team directly from within your app by clicking your user gravatar at the top right end corner of your application and hitting the Give us feedback entry in the menu. A form will show up letting you enter your name, email address, message to the Sifflet team, and even take a screenshot. You can use this to report bugs, share feedback about the application or even ask for help. The Sifflet team will follow up with you shortly after you submit your message!
Sifflet now supports key pair authentication for Snowflake, offering enhanced authentication security as an alternative to using a username and password.
To use key pair authentication, create the key pair by following the guide provided by Snowflake and then use the private key when adding the credentials to Sifflet as detailed in our documentation.
dbt build support
You can now use the dbt build command to generate the dbt artifacts to send to Sifflet, ensuring that you have full flexibility in configuring your dbt jobs. Refer to the dedicated documentation page for more details regarding our dbt Core integration.
🛠 Fixes
Fixed a behaviour where Monitors run with BigQuery repeated fields did not behave as expected
Fixed and Improved regexes generated via the AI suggestion feature. Fixed a case where the regex generated did not work correctly on Snowflake.
End-To-End Data Observability With Declarative Assets & Lineage
Sifflet already offers a large number of built-in integrations spanning your entire data pipelines’ stack. These integrations automatically collect metadata and lineage information and make it available in the Data Catalog.
However, there may be instances where you need to programmatically declare certain data pipeline assets. This is essential for achieving end-to-end data observability, especially for custom scripts, data applications, or assets from technologies not yet directly supported by Sifflet (e.g., Salesforce, SAP, Metabase, etc.).
The declarative framework now allows you to programmatically push any assets and lineage data to Sifflet. This enables you to document, govern, and visualize lineage across your entire data stack, regardless of asset type or technology. This new capability removes any limitations on the assets that can be integrated with Sifflet, paving the way for complete end-to-end data observability.
In the context of changes done for the declarative framework, you can now access assets' URI from the context menu located in the top right end corner of your asset pages.
Asset IDs that used to be in the Overview tab are now available in the same context menu, ensuring the asset page remains focused on the content that matters the most to users.
Finally, source IDs that also used to be in the Overview tab were entirely removed from asset pages and are now available on source pages.
