Snowflake Collected Data

This page lists what Sifflet imports from Snowflake once you connect your account.

Imported Assets

Sifflet imports the following objects from the schemas selected in the source:

Tables, including dynamic tables and external tables
Views, including materialized views
Streams

Collected Metadata

For tables, dynamic tables, and views:

Name, type, and description (Snowflake comment)
Column names, types, and descriptions
Tags on tables and columns, from SNOWFLAKE.ACCOUNT_USAGE.TAG_REFERENCES (classification tags and regular tags, not inherited tags)
View definition (SQL)
Last update time, from <DATABASE>.INFORMATION_SCHEMA.TABLES, used by freshness monitors
Data usage: which users read each table and how often, from SNOWFLAKE.ACCOUNT_USAGE.QUERY_HISTORY, ACCESS_HISTORY, and TABLES (Enterprise Edition or higher)
Asset update history (optional, disabled by default): the latest queries that modified each table, from SNOWFLAKE.ACCOUNT_USAGE.ACCESS_HISTORY and QUERY_HISTORY (Enterprise Edition or higher)

For streams:

Name, owner, creation date, and comment
Source table, stream type, and mode
Stale status and stale-after date
Stream definition (SQL)

Lineage

Sifflet builds table-level and field-level lineage between your Snowflake assets from:

SNOWFLAKE.ACCOUNT_USAGE.ACCESS_HISTORY (Enterprise Edition or higher): the objects and columns read and written by each query;
SNOWFLAKE.ACCOUNT_USAGE.OBJECT_DEPENDENCIES: dependencies of views, materialized views, streams, and external tables;
SNOWFLAKE.ACCOUNT_USAGE.DYNAMIC_TABLE_REFRESH_HISTORY and QUERY_HISTORY: dependencies of dynamic tables.
SNOWFLAKE.CORE.GET_LINEAGE: field-level lineage of dynamic tables, if the Sifflet role has the optional VIEW LINEAGE privilege.

When ACCESS_HISTORY returns no lineage (for example on Standard Edition), Sifflet parses the SQL of successful CREATE TABLE AS SELECT, CREATE VIEW, and INSERT ... SELECT queries from SNOWFLAKE.ACCOUNT_USAGE.QUERY_HISTORY instead.

Sifflet also links your Snowflake tables to the assets of your other integrations, such as dbt models, Airflow DAGs, and BI dashboards, to show end-to-end lineage.

Data Freshness

Sifflet refreshes Snowflake metadata on the source's schedule (see Integrations Management for the default frequency and scheduling).

Snowflake fills the ACCOUNT_USAGE views with a delay of up to 3 hours (see Snowflake Account Usage data latency). Lineage, tags, data usage, and update history collected from these views can therefore miss the queries and changes of the last few hours: they appear at the next refresh.

Limitations

Snowflake Standard Edition: ACCESS_HISTORY isn't available, so Sifflet doesn't collect data usage or asset update history, and builds lineage from the query history only.
Sifflet Native App for Snowflake: lineage isn't available, and new schemas aren't discovered automatically. See Sifflet Native App for Snowflake.

Data Exposure

Data preview and failing rows display rows of your Snowflake tables. Sifflet fetches them on demand and doesn't store them.
The asset update history, disabled by default, stores the SQL statements that modified your tables, which can contain sensitive values.

See Security for the data Sifflet accesses and stores.