✨ Feature Highlights

Freshness Improvements

Sifflet has in the past primarily been doing Freshness based on the distribution of the data, checking that data points with new dates were added. However this does not cover all freshness usecases, such as reloading a dimension table without a date field. In those cases what I want to check if if the data reload occurred within the usual time range !

New Template: Freshness (Update Time Gap)

This template can be applied to any table and will use the Table Metadata available to build up a history of typical time gaps between updates.

Due to the lack of history available, this monitor needs training time to be effective so you will need to wait a few data update cycles for a nice looking graph !

Supported Databases:

• Snowflake
• Bigquery
• Databricks
• Oracle
• Mysql

🛠 Fixes

• Fixed a bug that caused recurring data source update failures in some large environments.