Auto-Coverage

Setting up a complex monitor can take up to a few minutes - that can be very time-consuming, not even mentioning tedious. Here's where Sifflet Auto-coverage saved the day - it allows you to leverage the power of ML to generate a tailor-made setup on multiple tables in a matter of just a few clicks.

In the Catalog, use the Sifflet search engine to narrow down to a selection of tables on which Auto-Coverage needs to be applied. Select the tables and click the Auto-Coverage button.

Use the toggle to include/exclude any suggested data quality monitoring logic. There are currently 8 logics:

• unique if field is id: If a field has an ID format, it will create a rule to verify the unicity of the IDs
• not null if field is id: If a field has an ID format, it will create a rule to verify that the field does not contain any null values
• completeness if temporal: If a field has a time format, it will create a rule to verify that the volume of new ingested rows follows the expectations based on past behavior (ML-based rule).
• duplicate if temporal: If a field has a time format, it will create a rule to verify that there is no duplicate
• freshness if temporal: If a field has a time format, it will create a rule to verify that the data has been updated on the expected time
• schema change if always true: It will create a rule to detect any schema change
• email format if field is email: If a field has an email format, it will create a rule to verify that every value is under the appropriate format
• phone number format if field is phone number: If a field has a phone number format, it will create a rule to verify that every value is under the appropriate format

Once you click Scan, Sifflet will automatically identify the right fields to monitor and suggest a list of rules to create.

Use the toggle on the left to select the rules you want to be created, and use the dropdown lists on the right to define the severity of each rule.

To discover the way of creating customized rules, go to One-rule setup.

To understand how to finish configuring your rule, go to Time parameters.