You can integrate with dbt Cloud with the following steps:

1. (Optional) Create a dedicated Sifflet job
2. Generate an API Service Token with read-only access
3. Add the dbt Cloud integration in Sifflet

1 - (Optional) Create a dedicated Sifflet job

• Dedicated Sifflet job
  In order to generate all dbt artifacts for Sifflet, a dedicated job is recommended. In this job, you will only have to include the following commands, in this order:

1. dbt docs generate
2. dbt test

🚧

dbt commands order

Sifflet will look by default for the latest job that ran.
To get accurate data, the command dbt test must be the last one to run in the job in order to get a valid "run_results.json" artifact. For instance, this file gets overridden by a dbt run or dbt docs generate and will not contain any test run results.
Consequently, please make sure to unselect the option to "Generate docs" in your job configuration.

Please find below an example of correct job configuration on dbt:

• dbt Job scheduling configuration:
  On the Triggers section, select Schedule and provide relevant scheduling according to your needs.

📘

Scheduling frequency, timing & runtime

Be mindful to schedule this job in comparison with the other main jobs:

• At the same frequency
• After other jobs completed
• Consider the average run time of previous jobs to complete

2 - Generate an API Service Token with Read-Only access

🚧

The service token can only be generated with a dbt admin account

• Connect to dbt Cloud with a dbt admin account: https://cloud.getdbt.com
• On the left panel, Go to Account Settings > Service tokens
• Generate the new token by pressing "New Token":
  • Add the Permission Set "Read-Only" and the corresponding Project
  • After saving, the token will only be shown once, please store it in a safe location.

3 - Add the dbt Cloud integration in Sifflet

a. Add the token as a Sifflet secret

The token previously generated on dbt Cloud should be saved in Sifflet:

• On the left panel, choose "Integration" and then the submenu "Secrets"
• Press "New" Secret
• Secret format:

b. Add the dbt Cloud integration

In Sifflet, go to "Integration" --> "New", choose the dbt Cloud option.
The required parameters are:

• Name: choose a name to represent your dbt Cloud integration in Sifflet
• Base Url: by default https://cloud.getdbt.com/api/v2. You can change it in case your dbt Cloud instance is hosted in another location than North-America (see here for more details). For example, in case of EMEA, the url is https://emea.dbt.com/api/v2
• Account ID: your dbt Cloud account ID (see below on how to get it)
• Project ID: your dbt project ID (see below on how to get it)
• Job ID: your dbt job ID (see below on how to get it)
• Secret: the secret you just created in the previous step
• Frequency: how often Sifflet will sync with dbt Cloud. As mentioned above for the dbt Cloud schedule, consider adding a similar frequency and a CRON that will trigger the refresh after the dedicated Sifflet job is completed.

📘

How to get your dbt Cloud account ID, project ID and job ID

Connect to your dbt Cloud account and choose a specific project. The URL link should be similar to this: https://cloud.getdbt.com/#/accounts/12345/settings/projects/67890/

• The dbt Cloud account ID is 12345
• The project ID is 67890

In some cases, it is easier to create a dedicated job to generate the dbt artifacts.
You can then add the job ID in your connection.
For an URL https://cloud.getdbt.com/#/accounts/12345/projects/67890/jobs/223344/

• the job ID would be 223344

FAQ

The dbt tests listed in the monitor section do not show any status and last runs information

If the dbt tests listed in the monitor section do not contain information about the success or failure of the last run, it is likely due to the order between the commands docs generation and dbt test. Please ensure to run the commands in the following order and to deactivate the "Generate Docs?" from the execution settings:

1. dbt docs generate
2. dbt test

For more information, see step 1 above.