Airflow custom operators
In order to configure Airflow in Sifflet, you can refer to this page.
Additionally, you have at your disposal the below custom Airflow operators that enables you to interact programmatically with Sifflet.
For more details on how to install the Sifflet package for Airflow, please refer to this page
Please find below all the available operators:
SiffletRunRuleOperator
: Trigger one or several rulesSiffletDbtIngestOperator
: Send the dbt artifacts
SiffletRunRuleOperator
The operator takes as parameters:
task_id
: unique identifier of the taskrule_ids
: list of one or several rule Ids. For more details on how to find the rule_ids, please refer to this pageerror_on_rule_fail
(optional): default value is true. If the rule fails, the task will fail as well. You can choose to set the parameter to false for less critical rules.
sifflet_rule = SiffletRunRuleOperator(
task_id="sifflet_rule",
rule_ids=["<rule_id_1>","<rule_id_2>"],
)
SiffletDbtIngestOperator
The operator takes as parameters:
task_id
: unique identifier of the taskinput_folder
: directory of the dbt artifactstarget
: same value as in part 1, the target value of the profile that corresponds to your project (the "target" in your profiles.yml)project_name
: same value as in part 1, the name of your dbt project (the "name" in your dbt_project.yml file)
sifflet_rule = SiffletDbtIngestOperator(
task_id="sifflet_dbt_ingest",
input_folder="<DBT_PROJ_DIR>",
target="<TARGET>",
project_name="<DBT_PROJECT_NAME>",
)
Updated 7 months ago