Access Reference Assets Details for Improved Data Observability Coverage
You can now click reference assets on the lineage to view details such as the asset name. This makes it simpler to identify where these assets are located (i.e. in which database, schema, etc.) and consequently create the corresponding Sifflet data sources, enhance your overall data observability coverage.
We've introduced the capability to generate Monitor names and descriptions with AI. This will ensure you have highly descriptive descriptions and names for your monitors with low effort!
How? Simply select the AI icon in the text fields to write custom monitors. You will need to regenerate when monitors are now.
Tip: Not satisfied with the description, click it again and it will change!
Note: This generation is currently not aware of other descriptions and monitor names you have set on other monitors. Future versions aim to leverage these and make it even easier to standardise naming practices.
Need Help? Send Us a Message From Within the Sifflet App!
You can now reach out to the Sifflet team directly from within your app by clicking your user gravatar at the top right end corner of your application and hitting the Give us feedback entry in the menu. A form will show up letting you enter your name, email address, message to the Sifflet team, and even take a screenshot. You can use this to report bugs, share feedback about the application or even ask for help. The Sifflet team will follow up with you shortly after you submit your message!
Sifflet now supports key pair authentication for Snowflake, offering enhanced authentication security as an alternative to using a username and password.
To use key pair authentication, create the key pair by following the guide provided by Snowflake and then use the private key when adding the credentials to Sifflet as detailed in our documentation.
dbt build support
You can now use the dbt build command to generate the dbt artifacts to send to Sifflet, ensuring that you have full flexibility in configuring your dbt jobs. Refer to the dedicated documentation page for more details regarding our dbt Core integration.
🛠 Fixes
Fixed a behaviour where Monitors run with BigQuery repeated fields did not behave as expected
Fixed and Improved regexes generated via the AI suggestion feature. Fixed a case where the regex generated did not work correctly on Snowflake.
End-To-End Data Observability With Declarative Assets & Lineage
Sifflet already offers a large number of built-in integrations spanning your entire data pipelines’ stack. These integrations automatically collect metadata and lineage information and make it available in the Data Catalog.
However, there may be instances where you need to programmatically declare certain data pipeline assets. This is essential for achieving end-to-end data observability, especially for custom scripts, data applications, or assets from technologies not yet directly supported by Sifflet (e.g., Salesforce, SAP, Metabase, etc.).
The declarative framework now allows you to programmatically push any assets and lineage data to Sifflet. This enables you to document, govern, and visualize lineage across your entire data stack, regardless of asset type or technology. This new capability removes any limitations on the assets that can be integrated with Sifflet, paving the way for complete end-to-end data observability.
In the context of changes done for the declarative framework, you can now access assets' URI from the context menu located in the top right end corner of your asset pages.
Asset IDs that used to be in the Overview tab are now available in the same context menu, ensuring the asset page remains focused on the content that matters the most to users.
Finally, source IDs that also used to be in the Overview tab were entirely removed from asset pages and are now available on source pages.
Sifflet now integrates with Fivetran, allowing you to gain unparalleled visibility into your data's end-to-end journey, from source to destination.
With this release, the Sifflet lineage graph no longer starts at the data platform level. Instead, you now have access to two types of nodes that represent your data’s journey before it even reaches the data platform:
Fivetran connector nodes: These clickable nodes represent your Fivetran connectors, allowing you to get all the useful information about a given connector with just one click. This information includes the connector’s source and destination, its current status, sync frequency, and the timestamp of the latest sync. Accompanied by the Sifflet metadata that you’re accustomed to (owners, description, and last metadata refresh), this panel gives you, at a glance, all the necessary information about a given connector.
Source asset nodes: For every Fivetran connector, Sifflet will create nodes for the source assets from which Fivetran extracts data, giving you complete visibility on your data’s journey. These nodes, which allow you to get a truly end-to-end lineage within Sifflet, represent one of the key differentiators of our Fivetran integration.
End-to-end lineage via the Fivetran integration
By integrating your Fivetran account with Sifflet, you'll also be able to swiftly detect connector issues (in the Sifflet data catalog and via the Fivetran connector nodes in the lineage) and easily get a bird’s-eye view of your Fivetran connectors, no matter where you encounter them in the lineage.
To get started with the integration, refer to the dedicated documentation page. For additional information about the exciting features of this integration, check out the official release announcement.
🛠 Fixes
Fixed a bug that occasionally prevented Tableau and Looker assets from being connected to their upstream nodes in the lineage.
The incidents list page got revamped to improve the overall incident search user experience by making it consistent with the rest of the Sifflet application. By default, only Open and In progress incidents now show on the list, making it easier for you to focus on data quality issues that require immediate attention.
There are some monitors for which you don't necessarily want to create a Data Incident. You can now control Sifflet's automatic creation of an incident via the bulk edit from the Monitor list page.
This feature works also with dbt monitors.
In addition to the bulk edition of monitors, we've also made this toggle available at the dbt datasource level.
🛠 Fixes
Fixed Technical Error in some instances of Duplicate and Null monitors with Group By
Fixed Field Duplicate and Completeness monitors reporting incorrect results for empty datasets
Assigning the appropriate severity to a monitor is instrumental to ensure data quality issues are tackled with the right level of urgency by your teams. You can now bulk edit your monitors' severity, making it easier for you to update your monitor suite to ensure it is set to the proper criticality level.
