Auto-Coverage

Setting up one rule on a field or a table-level can take up to a few minutes.
This can be very time-consuming if you have hundreds of tables to monitor.
Auto-coverage allows you to set the most fundamental data quality rules on multiple tables in a matter of a few clicks.

In the Catalog, use Sifflet search engine to narrow down to a selection of tables on which Auto-Coverage needs to be applied. Move the toggle to use Multiple selection, select the tables by clicking them and then click the Auto-Coverage button.

Use the toggle to include/exclude any suggested data quality monitoring logic. There are currently 8 logics:

• unique if field is id: If a field has an ID format, it will create a rule to verify the unicity of the IDs
• not null if field is id: If a field has an ID format, it will create a rule to verify that the field does not contain any null values
• completeness if temporal: If a field has a time format, it will create a rule to verify that the volume of new ingested rows follows the expectations based on past behavior (ML-based rule).
• duplicate if temporal: If a field has a time format, it will create a rule to verify that there is no duplicate
• freshness if temporal: If a field has a time format, it will create a rule to verify that the data has been updated on the expected time
• schema change if always true: It will create a rule to detect any schema change
• email format if field is email: If a field has an email format, it will create a rule to verify that every value is under the appropriate format
• phone number format if field is phone number: If a field has a phone number format, it will create a rule to verify that every value is under the appropriate format

Once you click Scan, Sifflet will automatically identify the right fields to monitor and suggest a list of rules to create.

Use the toggle on the left to select the rules you want to be created, and use the dropdown lists on the right to define the severity of each rule.

To discover the way of creating customized rules, go to One-rule setup.

To understand how to finish configuring your rule, go to Time parameters.