Databricks Troubleshooting

This page lists common errors when you connect Sifflet to Databricks, and how to fix them. Sifflet shows these errors in the test report when you click Test, and in the logs of a failed refresh in the source details.

Connection

The Host Can't Be Resolved

Symptom

java.net.UnknownHostException: dbc-a1b2c3d4-e5f6.cloud.databricks.

Cause

The Host field doesn't contain a valid workspace hostname: it is truncated, misspelled, or contains a protocol or path.

Fix

1. In Databricks, copy the Server hostname from the Connection details tab of the SQL warehouse.
2. Paste it in the Host field of the source, without https:// and without a trailing /, for example dbc-a1b2c3d4-e5f6.cloud.databricks.com. See Step 2: Add the Databricks Source in Sifflet.

The Connection Times Out

Symptom

Generic Error: status: 504; status text: ; body: "\r\n504 Gateway Time-out\r\n\r\n504 Gateway Time-out\r\nnginx\r\n..."

Cause

Sifflet can't reach the workspace on the configured port: the Port field is wrong, or a network restriction blocks the traffic from Sifflet.

Fix

1. Set the Port field to the Port shown in the warehouse connection details, 443 by default.
2. If you use Databricks IP access lists, add the IP addresses of your Sifflet instance, listed in Settings > Integration Preferences. See Network Access.

Authentication

The Credential Isn't a Personal Access Token

Symptom

Invalid credentials type provided. Please ensure the credentials start with "dapi".

Cause

The Sifflet credential doesn't contain a Databricks personal access token. Sifflet doesn't support OAuth secrets or other credential types for Databricks.

Fix

1. Generate a personal access token for the service principal, as described in Step 1: Create a Service Principal in Databricks.
2. In Integrations > Credentials, edit the credential and paste the token as-is, with no quotes or extra spaces.

The Token Is Rejected by the Workspace

Symptom

Unauthorized access to Org

Cause

The token is expired or revoked, or it belongs to an identity that isn't a member of this workspace.

Fix

1. Check that the service principal is active and added to the workspace configured in the Host field.
2. Generate a new personal access token, as described in Step 1: Create a Service Principal in Databricks.
3. In Integrations > Credentials, update the credential with the new token.

The Token Doesn't Allow SQL Access

Symptom

Provided PAT token does not have required scopes: sql

Cause

The personal access token was created with scopes that don't include SQL access.

Fix

1. Generate a new personal access token that includes the sql scope, or no scope restriction.
2. In Integrations > Credentials, update the credential with the new token.

Permissions

Sifflet Can't Use the SQL Warehouse

Symptom

The test or the refresh fails with a permission error on the SQL warehouse, or the queries never start.

Cause

The service principal doesn't have the Databricks SQL access entitlement or the Can use permission on the SQL warehouse, or the warehouse configured in Http Path doesn't exist anymore.

Fix

1. Check that the Http Path field matches the HTTP path of an existing warehouse.
2. Grant the entitlement and the warehouse permission, as described in Step 1: Create a Service Principal in Databricks.

Catalog Permissions Are Missing

Symptom

Missing required catalog permission: SELECT on TABLE samples.information_schema.column_tags

Failed to check catalog permission: USE on DATABASE samples

Cause

The service principal doesn't have USE CATALOG, USE SCHEMA, and SELECT on the catalog in scope. Sifflet reads metadata from the catalog's information_schema, so it needs these privileges at the catalog level. The Failed to check variant means that Sifflet couldn't even list the grants on the object, for example because the catalog doesn't exist or isn't visible to the service principal.

Fix

1. Grant the catalog privileges, as described in Grant the Permissions.
2. Click Test again on the source.

Lineage Permissions Are Missing

Symptom

Missing required lineage permission: SELECT on TABLE system.access.column_lineage

Failed to check lineage permission: USE on SCHEMA system.access

Cause

The service principal can't read the system.access.column_lineage system table, or the system.access schema isn't enabled in your Databricks account. Sifflet requires this access to build lineage, and the refresh fails without it.

Fix

1. Ask a Databricks account admin to enable the system.access system schema. See Other Prerequisites.
2. Grant the lineage privileges, as described in Grant the Permissions.

Warning About pipeline_update_timeline

Symptom

The test succeeds with this warning:

Your configuration lacks the SELECT permission on system.lakeflow.pipeline_update_timeline. This permission would make it possible to link Databricks Jobs to tables if you are using Databricks Pipelines to transform your data in Databricks (and you have created a Databricks Jobs integration in Sifflet).

Cause

The optional permission that links tables written by Lakeflow Declarative Pipelines to Databricks jobs isn't granted. The rest of the integration works.

Fix

1. If you use Lakeflow Declarative Pipelines and the Databricks Workflows integration, grant the optional job lineage privileges described in Grant the Permissions.
2. Otherwise, you can ignore the warning.