Data Products

Overview

The term 'data product' has a range of definitions across the industry. Within Sifflet, we define a Data Product as a logical grouping of multiple data assets (like datasets, dashboards, and transformation tasks) that collectively serve a common business use case. This approach provides a flexible framework for you to implement your own definition of a data product while gaining powerful new capabilities for monitoring, ownership, and governance.

By grouping assets into Data Products, you can move from monitoring individual tables and dashboards to monitoring the health of an entire end-to-end pipeline or business-critical initiative at a glance.

Key Benefits of Using Data Products:

• Holistic Monitoring: Understand the health of an entire data pipeline (e.g., "Product Analytics Pipeline") instead of tracking dozens of individual assets separately.
• Clear Ownership: Assign clear owners to specific business use cases, simplifying reporting and accountability at the team level.
• Value-Driven Insights: Directly connect technical data assets to the business value they generate (e.g., "This Reverse ETL pipeline costs $X and supports $Y in sales operations").
• Governance at Scale: Standardize alert notifications for your critical use cases by defining default notification channels at the product level. This ensures all alerts from the data product's assets are consistently routed to the right stakeholders, streamlining communication and incident response.

Creating and Managing Data Products

System Admins and Domain Editors can create and manage Data Products. All other users can view Data Products, subject to their existing domain-level permissions.

Creating a Data Product

1. Navigate to Data Products from the main menu in the Sifflet platform.

2. Click the New Data Product button to open the configuration page.

3. Fill in the Data Product Details:

   • Domain: The Domain this product belongs to. Every data product belongs to one domain in Sifflet, but you can also use the default "All" domain.

     📘

     Once the data product is created, its domain is no longer changeable.

   • Name: A unique and descriptive name for the Data Product (e.g., "Marketing Campaigns Analysis").
   • Description: A detailed explanation of the use case this product serves.
   • Owners: The Sifflet users responsible for this Data Product.
   • Tags: Add relevant tags for filtering and organization.
   • SLA: Define the Service Level Agreement for the data delivered by this product (e.g., "Data must be updated by 08:00 AM daily").
   • Add Assets: In the second section, select the assets (datasets, dashboards, pipelines, etc.) that constitute this Data Product. You can either select the assets one by one or leverage the filtering conditions for dynamic at-scale selection.
   • Configure Notifications: In the final section, you can configure two types of notifications for your Data Product:

   Status Change Notifications : Receive alerts whenever the status of the Data Product changes.

   🟢 Healthy → 🟡 At Risk

   🟢 Healthy → 🔴 Critical

   🟡 At Risk → 🔴 Critical

   Summary Notifications : Receive periodic summaries of activity and events related to the Data Product.

   Hourly: Runs at the beginning of every hour (e.g., 10:00, 11:00, 12:00).

   Daily: Runs once per day at the selected hour.

   Weekly: Runs once per week on the selected day and at the selected hour.

   Message example

aDataproduct — 🔴 Critical

🔴 1 critical incident
Data product critically - 1 failing monitor

🔵 1 low incident
 Data product low Incident - 1 failing monitor

The Data Products List

The main Data Products page provides a comprehensive list of all products in your environment. The table includes:

• Data Product Name: The name of the product.
• Status: Health status of the product base on incidents
• Number of Assets: The count of assets included in the product.
• Owner(s), SLA, Description: Key metadata for quick reference.

Understanding Data Product Status

The health of a Data Product is determined by the severity of its active incidents and provides a high-level view of the overall reliability of the product.

Each Data Product is assigned one of three statuses:

🟢 Healthy: No active incidents are affecting the Data Product.

🟡 At Risk: The Data Product has one or more active moderate incidents that may impact data quality or reliability. Only low and moderate incidents

🔴 Critical: The Data Product has one or more active high-risk or critical incidents requiring immediate attention. Moderate ,High-risk and critical incidents

Viewing Data Product Details

Clicking on a Data Product from the list will open a side panel with two tabs:

1. Overview: Displays all the metadata you configured, such as owners, SLA, and tags.
2. Assets: Lists all the data assets that are part of the Data Product. Clicking on an asset will take you to its dedicated page.
3. Incident List: Quickly identify and investigate issues impacting your Data Products.

Integration Throughout Sifflet

To make Data Products a core part of your workflow, they are integrated into other areas of the platform:

• Asset Page: For any asset that belongs to a Data Product, you will see a "Data Products" attribute in its Overview tab, linking directly to the relevant product(s).