Athena Collected Data

This page lists what Sifflet imports from Amazon Athena once you connect Sifflet to Athena.

Imported Assets

Sifflet imports the following assets into the Data Catalog, for each Athena database in the scope of the source:

• Tables (shown as external tables, since Athena tables point to data stored in Amazon S3)
• Views

Collected Metadata

Lineage

Sifflet builds lineage from three sources:

• Table definitions: each table is linked to the S3 location it reads its data from.
• View definitions: each view is linked to the tables and views it selects from.
• Query history (optional, requires the lineage permissions): Sifflet reads the successful DDL and DML queries of the source's workgroup that ran in the scanned database and data catalog, for example CREATE TABLE AS SELECT and INSERT INTO statements, and links the tables they write to the tables they read from.

Sifflet also links Athena tables to the assets of other integrations that read from them, such as QuickSight datasets.

Data Freshness

Sifflet refreshes Athena metadata and lineage on the schedule of the source: see Integrations Management for the default frequency and how to change it. At each refresh, Sifflet reads recent queries from the workgroup's query history, which Athena keeps for 45 days.

Limitations

• Sifflet only reads the query history of the workgroup configured in the source: queries run in other workgroups don't contribute to lineage.

Data Exposure

Sifflet reads row-level data from Athena only on demand, for data preview and to display failing rows of a monitor, and doesn't store it. Query results, including those of preview and monitor queries, are written by Athena to the S3 output location you configure, in your AWS account. Sifflet reads the text of the queries in your workgroup's history to build lineage. See Security for details on the data Sifflet accesses and stores.