Snapshot Mode
Monitor data assets without time fields.
Overview
Snapshot Mode is a monitoring mode that doesn't use a Time Field, instead generating time-information based on data snapshots taken at regular intervals. Thanks to this feature, it's possible to create ML-based monitoring on tables that don't have any time fields.
How to
How to use
The ML models will be taking snapshots of the data at every monitor run and computing a time series that will later be used to train the models and detects anomalies. With this feature, users can monitor more than just the metadata of dimension tables with no time fields, such as a sudden increase of formatting issues in the customers emails field, or a sudden drop of the average lifetime value of your customers.
How to activate
In a Monitor Editing view, leave the Time Window
box unchecked during parameters configuration.
Availability
Snapshot Mode is available for ML-based (dynamic-threshold) monitors only.
For SQL monitors, leaving the Time Window box unchecked will also lead to running the analysis using the whole dataset, however, it won't trigger generation of snapshots of data.
Training time
Sifflet ML models require at least 10 data points to establish a reliable training sample. Anomalies detection won't start until that threshold is met (first 10 Monitor Runs).
Updated 9 months ago