Auto-Monitoring (Sentinel) ᴮᴱᵀᴬ

Sentinel is Sifflet's AI-powered agent designed to automate and streamline the creation of data quality monitors. It addresses the challenge of manually setting up comprehensive monitoring by intelligently analyzing data assets to suggest relevant, ready-to-use monitors.

Whether you are monitoring a single table, multiple assets in bulk, or an entire data product, Sentinel helps ensure complete data quality coverage with minimal effort.

Key Benefits

• Save Time and Effort: Go from a new data asset to a fully monitored one in just a few clicks. Eliminate the manual work of configuring individual monitors.
• Improve Monitoring Coverage: Sentinel discovers potential data quality issues you might have missed, suggesting monitors for data formats, value ranges, logical consistency, and more.
• Scale with Ease: Apply monitoring best practices to dozens of assets at once from the data catalog or ensure coverage for all assets within your data products.
• Leverage AI-Powered Insights: Sentinel uses the context of your data—column names, metadata, and actual data samples—to recommend the most effective monitors for your specific use case.

Three Ways to Use Sentinel AI

Sentinel is designed to fit your workflow. You can generate recommendations for a single asset, a custom selection of assets, or an entire data product.

1. On a Single Data Asset

This approach is perfect for when you are working with a specific table and want to quickly ensure it's monitored correctly.

• Step 1: Start the Analysis: Navigate to the asset page and click the “Auto-Monitoring” button.
• Step 2: Let Sentinel Work: An analysis window will appear, showing Sentinel's real-time progress.
• Step 3: Review and Select: In under 30 seconds, you will receive a list of suggestions. Each includes an explanation of why the AI recommended it. Use the checkboxes to select the monitors you want.
• Step 4: Configure and Create: Apply a run schedule and notification settings, then click "Create monitors" to create all selected monitors at once.

2. In Bulk from the Data Catalog

This method is ideal for applying consistent monitoring across multiple assets without having to configure each one individually.

❗️

Limitation regarding the number of assets

Currently, you can select a maximum of 10 assets at a time for bulk analysis from the Data Catalog.

• Step 1: Select Your Assets: Go to the Data Catalog. Use the checkboxes to select all the data assets you wish to monitor.

• Step 2: Begin Bulk Analysis: Once you've selected your assets, the “Auto-Monitoring” button will become available. Click it to start.

  

• Step 3: Track Progress: Sentinel will analyze each asset. You can track the progress individually and review recommendations as they become available—no need to wait for the entire batch to finish.

  

• Step 4: Review and Create: Browse the recommendations for all assets. Select the monitors you want, apply universal settings, and create them all in a single action.

3. Across an Entire Data Product

This is the most powerful way to ensure comprehensive data quality for a curated collection of assets.

• Step 1: Navigate to Your Data Product: Open the Data Product you want to monitor.
• Step 2: Start the Analysis: Click the “Auto-monitoring” button on the data product side-panel. Sentinel will automatically begin analyzing every data asset included in that product.
• Step 3: Review as Results Arrive: Just like the bulk catalog option, you can see results for each asset as soon as its analysis is complete.
• Step 4: Create with Confidence: Select the monitors that make sense for your data product, configure your schedule and alerting, and apply comprehensive monitoring with one click.

What Kind of Monitors Can Sentinel Recommend?

Sentinel is equipped to suggest a wide variety of monitors to cover all dimensions of data quality.

• Format & Pattern Validation: Checks for standard formats like Email or UUID and custom patterns like ORD-XXX using regex.
• Data Integrity & Consistency: Recommends uniqueness checks for ID columns, ensures values in a column belong to an expected list (e.g., "Status"), and validates logical rules (e.g., $shipping_date > $order_date).
• Numerical & Statistical Health: Suggests monitors for logical value ranges (e.g., $price > 0) and detects significant changes in the statistical distribution of your data.

Frequently Asked Questions (FAQ)

Q: How long does the analysis take?
For a single asset, recommendations are typically ready in under 30 seconds. For bulk analysis, the time will vary, but you can review results for each asset as they become available without waiting for the entire job to complete.

Q: Will Sentinel recommend monitors that I have already created?
No. Sentinel checks for existing monitors on an asset and will not suggest duplicates.

Q: How does the review process work when analyzing multiple assets at once?
When you run Sentinel on multiple assets, you don't have to wait for the entire process to finish. Recommendations will appear for each asset as its analysis is completed, allowing you to start reviewing and selecting monitors while others are still in progress.

Q: Can I edit a recommended monitor before creating it?
The feature is designed for rapid, bulk creation with shared settings. While you can't edit fine-grained details during the recommendation step, you can easily edit one or multiple monitors from the monitoring page after they have been created.