Looker

Sifflet connects to your Looker instance to show which dashboards and explores depend on the tables you monitor in your data warehouse. When an incident affects an upstream table, you see right away which Looker dashboards are impacted and how much they are used, and from any dashboard you can trace its data through explores and views back to the warehouse tables that feed it. With the Sifflet Insights browser extension, Looker users see this data quality status directly in Looker.

See Looker Collected Data for the full list of assets and metadata Sifflet imports.

Permissions Required

Sifflet has read-only access to Looker. It authenticates to the Looker API 4.0 with the API key (client ID and client secret) of a dedicated Looker user, whose role only contains read permissions. Optionally, Sifflet also reads your LookML project from its Git repository with a read-only token or deploy key (see Step 3).

In Looker, the Sifflet user has a role made of a permission set and a model set:

In your Git provider (optional, for Step 3):

API load: at each refresh, Sifflet calls the Looker API to read dashboards and explores, and queries System Activity. Sifflet refreshes BI sources weekly by default to limit this load.

Requirements

Required Roles for Setup

• In Looker: the Admin role, to create the permission set, the role, the Sifflet user, and its API key.
• In your Git provider (optional, for Step 3): the rights to create an access token or a deploy key with read access to the LookML repository.
• In Sifflet: the Admin role to create the credentials, and the Admin or Editor role to create the source.

Supported Versions and Editions

Sifflet supports Looker instances that expose the Looker API 4.0: Looker (original), Looker (Google Cloud core), and customer-hosted Looker instances. Looker Studio (formerly Google Data Studio) is not supported.

For the optional LookML repository, Sifflet supports any Git server reachable over HTTPS or SSH, such as GitHub, GitLab, and Bitbucket, including their self-hosted editions.

Network Access

• Default: Sifflet connects to the Looker API over HTTPS from the internet, so your Looker API endpoint must be reachable from the public internet.
• IP allowlisting: if you restrict access to Looker with an IP allowlist, allow the IP addresses of your Sifflet instance, listed in Settings > Integration Preferences.
• LookML repository (optional): Sifflet clones the repository from the internet, over HTTPS or SSH. A self-hosted Git server must be reachable from Sifflet, and you can restrict its access to the IP addresses of your Sifflet instance.

Other Prerequisites

• To see lineage from Looker to your data warehouse, add the warehouse tables to Sifflet first through their own source. See Lineage.

Connect Sifflet to Looker

Step 1: Create an API Key in Looker

Create a Permission Set and a Role

1. In Looker, go to Admin > Roles and click New Permission Set.

2. Enter a name (for example Sifflet) and select the following permissions:

   Text

   access_data
   see_lookml_dashboards
   see_looks
   see_user_dashboards
   explore
   see_lookml
   see_system_activity

   To collect the email addresses of dashboard creators and last updaters, also select see_users (optional).

   

3. Click New Permission Set to save it.

4. Still in Admin > Roles, click New Role, enter a name (for example Sifflet), select the permission set you created, select a model set that contains all the models to catalog (for example All), and save the role.

   

Create the Sifflet User

1. Go to Admin > Users and click Add Users.

2. Enter an email address for the Sifflet user, select the role you created, and uncheck Send setup emails. The email address doesn't need to be a real mailbox: use an internal address for bookkeeping, or a generic one such as sifflet_user@<YOUR_DOMAIN>.com.

   

3. Give the Sifflet user View access to every folder that contains dashboards to catalog, as described in Looker's documentation on folder access. Dashboards in folders the user can't view aren't imported.

Generate the API Key

1. In Admin > Users, click Edit next to the Sifflet user.

2. In API Keys (API3 Keys in older Looker versions), click Edit Keys, then New API Key.

3. Copy the Client ID and the Client Secret.

   

You now have the client ID and client secret needed for step 2.

Step 2: Add the Looker Source in Sifflet

1. Create the credential: in Integrations > Credentials, click New credential, enter a name, and paste the following JSON, replacing the placeholders with the client ID and client secret:

   JSON

   {"user": "<CLIENT_ID>", "password": "<CLIENT_SECRET>"}

2. Create the source: in Integrations > Sources, click New source, select Looker, and fill in the following fields:

   Field	Description	Example
   Source name	A name for the source in Sifflet	Looker
   Host	The URL of your Looker instance, followed by /api/4.0, without a trailing slash	https://abcdef.cloud.looker.com/api/4.0
   Credential	The credential you created above	looker-api-key

   If the connection fails because of the port, add the API port to the host: https://abcdef.cloud.looker.com:443/api/4.0 for instances created after July 7, 2020, or https://abcdef.cloud.looker.com:19999/api/4.0 for older instances (see Looker's API documentation).

   Leave the Git configuration empty for now: you add it in Step 3.

3. Scope: Sifflet catalogs all the dashboards and explores that the Sifflet user can access. To exclude content, restrict the user's folder access or model set in Looker. See Integrations Management for the refresh frequency and the other generic source settings.

4. Test and save: click Test to check that Sifflet can sign in to the Looker API. If the test fails, click Display report to see the details. Then click Save & Run. Sifflet starts a first refresh: follow its status and logs in the source details (click the source name in Integrations > Sources). Looker assets appear in the Data Catalog when the refresh completes.

Step 3 (Optional): Connect Your LookML Repository

Without the repository, Sifflet only reads user-defined dashboards and the explores they use through the Looker API, and links each explore to the table of its base view. With the repository, Sifflet also parses the LookML files of your project (.model.lkml, views, and dashboards) to import LookML dashboards, which the Looker API doesn't list, and to build complete lineage from all the views of your explores, including joined views and SQL derived tables, to your warehouse tables.

You can add several repositories to the same Looker source, one per LookML project. Sifflet reads each repository at each refresh and never writes to it.

Sifflet supports three authentication methods. The access token method is recommended because it's the simplest to set up and to restrict to read-only access.

Recommended: Access Token

1. Create an access token with read access to the repository:

   Git provider	Token and permissions
   GitHub	A fine-grained personal access token restricted to the repository, with the Contents repository permission set to Read-only; or a classic personal access token with the repo scope
   GitLab	A personal, project, or group access token with the read_repository scope. Project and group access tokens need at least the Reporter role.
   Bitbucket	An HTTP access token with the Project read (or Repository read) permission

2. In Sifflet, go to Integrations > Credentials, click New credential, and enter the token in the format that matches your Git provider:

   • GitHub: a JSON object with a username and the token. The username can be the user that owns the token or any non-empty string.

     JSON

     {"user": "<GITHUB_USERNAME>", "password": "<TOKEN>"}

   • GitLab:

     JSON

     {"password": "<TOKEN>"}

   • Bitbucket: the Authorization header value, as plain text:

     Text

     Bearer <TOKEN>

3. Go to Integrations > Sources, open the Looker source, click Edit, then click Add git configuration, and fill in the following fields:

   Field	Description	Example
   URL	The HTTPS clone URL of the repository, ending in .git	https://gitlab.com/<GROUP>/<LOOKML_PROJECT>.git
   Credential	The credential you just created	lookml-repo-token
   Branch	The branch Sifflet reads, usually your Looker production branch	main
   Authentication type	USER_PASSWORD for GitHub and GitLab, HTTP_AUTHORIZATION_HEADER for Bitbucket	USER_PASSWORD

   

   

4. Click Test Connection in the Git configuration to check that Sifflet can reach the repository, then save the source.

⚠️ Access tokens expire according to your Git provider's settings. Before the token expires, create a new one and update the credential in Integrations > Credentials.

Alternative: SSH Key

1. Generate an SSH key pair in PEM format, without a passphrase (Sifflet doesn't support passphrase-protected keys):

   • For GitHub, use an ECDSA key:

     Shell

     ssh-keygen -t ecdsa -m pem -b 521 -C <YOUR_EMAIL> -f <KEY_NAME> -N ""

   • For GitLab and Bitbucket, use an RSA key:

     Shell

     ssh-keygen -t rsa -m pem -b 4096 -C <YOUR_EMAIL> -f <KEY_NAME> -N ""

   The command creates the private key <KEY_NAME> and the public key <KEY_NAME>.pub in the current directory. The email in the comment is only used to identify the key.

2. Add the public key (<KEY_NAME>.pub) to the repository as a read-only deploy key: see the documentation of GitLab, GitHub, or Bitbucket.

3. In Sifflet, go to Integrations > Credentials, click New credential, and paste the full content of the private key file <KEY_NAME>, including the first and last lines:

   Text

   -----BEGIN EC PRIVATE KEY-----
   <PRIVATE_KEY_CONTENT>
   -----END EC PRIVATE KEY-----

   An RSA key starts with -----BEGIN RSA PRIVATE KEY----- instead.

4. Go to Integrations > Sources, open the Looker source, click Edit, then click Add git configuration, and fill in the following fields:

   Field	Description	Example
   URL	The SSH clone URL of the repository	[email protected]:<GROUP>/<LOOKML_PROJECT>.git
   Credential	The credential you just created	lookml-repo-ssh-key
   Branch	The branch Sifflet reads, usually your Looker production branch	main
   Authentication type	SSH	SSH

   

   

5. Click Test Connection in the Git configuration to check that Sifflet can reach the repository, then save the source.

Troubleshooting

See Looker Troubleshooting for sign-in errors, missing dashboards, and LookML repository issues.