You can integrate with dbt Cloud with the following steps:
- Generate an API Service Token
- Add the dbt Cloud integration in Sifflet
The service token can only be generated with a dbt admin account
- Connect to dbt Cloud with a dbt admin account: https://cloud.getdbt.com
- On the left panel, Go to Account Settings > Service tokens
- Generate the new token by pressing "New Token":
- Add the Permission Set "Read-Only" and the corresponding Project
- After saving, the token will only be shown once, please store it in a safe location.
The token previously generated on dbt Cloud should be saved in Sifflet:
- On the left panel, choose "Integration" and then the submenu "Secrets"
- Press "New" Secret
- Secret format:
In Sifflet, go to "Integration" --> "New", choose the dbt Cloud option.
The required parameters are:
- Name: choose a name to represent your dbt Cloud integration in Sifflet
- Account ID: your dbt Cloud account ID (see below on how to get it)
- Project ID: your dbt project ID (see below on how to get it)
- Job ID: your dbt job ID (see below on how to get it)
- Secret: the secret you just created in the previous step
- Frequency: how often Sifflet will sync with dbt Cloud.
How to get your dbt Cloud account ID, project ID and job ID
Connect to your dbt Cloud account and choose a specific project. The URL link should be similar to this: https://cloud.getdbt.com/#/accounts/12345/settings/projects/67890/
- The dbt Cloud account ID is 12345
- The project ID is 67890
In some cases, it is easier to create a dedicated job to generate the dbt artifacts.
You can then add the job ID in your connection.
For an URL https://cloud.getdbt.com/#/accounts/12345/projects/67890/jobs/223344/
- the job ID would be 223344
- Dedicated Sifflet job
In order to generate all dbt artifacts for Sifflet, a dedicated job is recommended. In this job, you will only have to include the following commands, in this order:
dbt docs generate
- dbt Job configuration:
On your dbt job > "Settings", you can see on the bottom the schedule of the job. Be mindful of it (and the average running time) when choosing the sync frequency on Sifflet.
- dbt commands order:
Sifflet will look by default for the latest job that ran.
To get accurate data, the command
dbt testmust be the last one to run in the job in order to get a valid "run_results.json" artifact. For instance, this file gets overridden by a
dbt docs generateand will not contain any test run results.
Consequently, please make sure to unselect the option to "Generate docs" in your job configuration.
Please find below an example of correct job configuration on dbt:
Updated about 1 month ago