Asset History

The asset update history feature provides a detailed, 7-day log of the queries that interact with your data and the operations performed on your assets, helping you understand usage patterns, track changes, and debug issues faster.

🚧

Beta Feature

This feature is still in beta. Please contact your Sifflet representative to request access for an early trial.

Key Benefits

Richer Asset Context

Asset History is organized into two powerful tabs that provide a complete picture of your data's lifecycle:

• 📈 Popular Queries Tab: Understand how your assets are being used. This tab showcases the most common queries that have read data from the asset in the past week, helping you identify key dependencies, usage patterns, and downstream consumers.
• ✍️ Update Queries Tab: Track every change made to your asset. This tab lists all queries and jobs that have updated the asset or ingested new data into it. This is essential for debugging ingestion failures, understanding data lineage, and monitoring modifications.

Powering Sifflet's Intelligence Engine (Coming Soon)

Beyond just displaying query information, the data collected by Asset History is foundational for Sifflet's advanced AI capabilities:

• Automated Root Cause Analysis: By correlating data quality incidents with the history of update queries, our Root Cause Analysis AI Agent, Sage, can automatically pinpoint the exact query failure or code change that impacted an asset's health, dramatically reducing investigation time.
• Smarter Monitor Recommendations: Our AI Agent, Sentinel, analyzes popular query patterns to understand how an asset is truly used. This allows Sentinel to recommend more intelligent, context-aware data quality monitors that focus on the columns and logic that matter most to your business.

⚠️ Important: Data Privacy and Security

At Sifflet, we take data security and privacy seriously. This feature is disabled by default on all Sifflet instances for a critical reason: it involves storing the full text of SQL queries, which may contain sensitive information, not just metadata.

For example, a standard write operation could expose personally identifiable information (PII):

INSERT INTO MY_PROD_DATABASE.MY_PROD_SCHEMA.USERS (first_name, last_name, email)
VALUES ('Jean', 'Dupont', '[email protected]');

This query would be extracted from your data platform and stored in Sifflet to be displayed in the asset's history. While your Sifflet instance is hosted in a highly secure and isolated database, ingesting this type of data may not align with your organization's data privacy policies.

Action Required: Before enabling this feature, please consult with your Data Protection Officer (DPO) or security team to ensure it complies with your company's data governance standards.

For more information, please refer to our Security and Compliance documentation.

How to Enable Asset History

The feature can be enabled by an administrator from the Sifflet settings:

1. Navigate to Settings > Integration Preferences.
2. Locate the "Enable source history ingestion" section.
3. Toggle the switch to enable the feature.

How It Works

• The 7-day query history is collected and refreshed each time the parent Sifflet Source is refreshed.
• For more details on source refresh frequency, refer to the dedicated page.

Supported Platforms

• BigQuery
• Snowflake (Enterprise plan)

Limitations and Quotas

To ensure optimal performance, the feature is governed by the following quotas:

• Maximum Operations per Asset: A limit of 1,000 operations is stored for each asset. For heavily used assets, only the most recent operations are retained.
• Maximum Statement Length: The details for each operation (e.g., the SQL statement) are limited to 4,000 characters. Longer statements will be truncated.

Future versions of this feature will include the ability to fetch the full text of longer SQL statements directly from the UI.