Data Pipelines
We refer by Data Pipelines to all the operations, jobs, and assets that participate in the data flow within the data stack. These may include:
- A transformation operation that generates a table (example: a dbt model)
- An orchestration operation that triggers a transformation
- A data movement operation that takes data from one system to another
Integrating Sifflet with your data pipelines allows you to enrich your lineage graph with pipeline metadata, access up-to-date pipeline status within Sifflet, and have additional context when debugging data incidents.
Sifflet currently integrates with the following data pipelining tools:
- Apache Airflow (and its cloud variations: MWAA and Cloud Composer)
- dbt (both dbt Core and dbt Cloud)
- Fivetran
Don't see all of your tools on the list? Reach out to us and we'd be happy to discuss your use case!
Updated 16 days ago