Looker

You can integrate with Looker to map the dependencies between Looker objects and data pipelines.
Sifflet search includes all Looker dashboards.

To connect Sifflet to Looker, you need a Looker account with admin rights to create read-only access for Sifflet.
The main steps are the following:

1. Looker API Key: Generate an API Key for Sifflet's read-only access to Looker's API and add it to Sifflet
2. Connect Sifflet to your LookML Repository (with an access token or an SSH key)

1. Generate a Looker API Key

Before generating the API Key, you will need to create a dedicated role with permissions and create a dedicated user with this role.

a. Create a Read-Only Permission Set and a Dedicated Role

• In "Admin" -> "Roles", create a "New Permission Set" (for instance "SiffletRole").
  The list of permissions needed is:

  access_data
  access_data.see_lookml_dashboards
  access_data.see_looks
  access_data.see_looks.see_user_dashboards
  access_data.see_looks.explore
  access_data.see_looks.see_lookml
  see_system_activity

• In "Admin" -> "Roles", create a "New Role". Select the newly created Permission Set

b. Create the User with This Read-Only Role

• In "Admin" -> "Users" -> "Add Users" to create a dedicated Sifflet user. Select the Role you created previously.

  The email doesn't need to be valid. You can choose an internal email for bookkeeping or a generic one such as sifflet_user@<domain>.com
  You can also uncheck the "Send setup emails" option.

c. Generate the API Key

You can now add the API Key to the Sifflet user:

• Once the Sifflet user is created, in "Admin" -> "Users", select "Edit" next to the newly created user
• In "API3 Keys", select "Edit Keys" then "New API3 Key"
  This will generate a "Client ID" and a "Client Secret" that you will need to add in Sifflet.

d. Add the Looker Data Source in Sifflet

1. Create the Looker credential that Sifflet uses to connect:
   • In Sifflet, go to Integrations > Credentials
   • Click New credential
   • Information required:
     • Name of the credential
     • The credential format is the following:

{"user":"ClientID","password":"ClientSecret"}

2. Add the Looker data source:
   • Go to Integrations > Sources
   • Click New source
   • Information required:
     • Name of the data source
     • Type: "Looker"
     • Host: URL of the Looker site and append at the end the following "/api/4.0".
       For instance if you usually connect to Looker on "https://abcdef.cloud.looker.com/
       ", then you should add the following on host: "https://abcdef.cloud.looker.com/api/4.0"
     • Secret: choose the newly created credential
     • The rest of the fields are optional, they are used for the SSH connection to your LookML repository (see here)

If a port issue arises when connecting, add ":443/api/4.0" or ":19999/api/4.0" instead.
It depends on the date of creation before or after 07/07/2020 (see Looker API doc), the most recent being 443.

2. Connect Sifflet to Your LookML Repository with an Access Token or an SSH Key

You can grant access to your LookML Repositories with a read-only token or SSH Key.

👍

Several repositories for one Looker connection

You can add several repositories to your Looker data source if needed.

Connect with an Access Token

a. Generate an Access Token

You can follow the official documentation here: GitLab, GitHub, Bitbucket.
The token only needs read access:

b. Configure Sifflet with the Token

1. Create the credential that contains the token that Sifflet uses to connect.

   • In Sifflet, go to Integrations > Credentials

   • Click New credential

   • Information required:

     • Name of the credential

     • The credential format is the following:

       • For GitHub: {"user": "<user_for_the_token>", "password":"<token>"}
         For <user_for_the_token> you can choose the user associated with the token or any string of characters, as long as you do not leave it empty
       • For GitLab: {"password":"<token>"}
       • For Bitbucket: Bearer <token>

2. Edit the Looker data source you created in part 1 to finalize the connection to your LookML repository:

   • Go to Integrations > Sources, choose your data source, and then click Edit
   • Click on Add Git Configuration

• Information required:
  • URL: the URL of your repo. It should have the following format https://###.git
  • Secret: the credential you just created
  • Branch (optional): the default branch if left empty. You can also specify the branch you want Sifflet to sync with
  • Auth Type:
    • use USER_PASSWORD if connecting with GitHub/GitLab
    • use HTTP_AUTHORIZATION_HEADER if connecting with Bitbucket
      Once all information is filled, you can test your token connection with the Test Connection button

Or Connect with an SSH Key

Another alternative to connect Sifflet to your LookML repository is to generate an SSH key.
The main steps are the following:

a. Generate an SSH Key
b. Configure your repo (GitLab, Bitbucket, etc.) with the public key
c. Configure Sifflet with the private key

a. Generate an SSH Key

• You can generate one in your local machine by running on your terminal the following command:
  • For GitHub: ssh-keygen -t ecdsa -m pem -b 521 -C [email protected]
  • For GitLab/Bitbucket: ssh-keygen -t rsa -m pem -b 4096 -C [email protected]
    The email doesn't need to be an existing one, but use your own for tracing purposes.
• You will then have to input the information required on the terminal prompt:
  • File location: no condition
  • Passphrase: leave it empty
    The prompt will be similar to the below prompt:

sifflet % ssh-keygen -t rsa -m pem -b 4096 -C [email protected]
Generating public/private rsa key pair.
Enter file in which to save the key: : <name_of_key>
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in <name_of_key>
Your public key has been saved in <name_of_key>.pub
The key fingerprint is:
SHA256:nt.... [email protected]
The key`s randomart image is:
+---[RSA 4096]----+
#################
+----[SHA256]-----+

• The pair of private/public keys will be created in your current directory: <name_of_key> and <name_of_key>.pub

b. Configure Your Repo

With the public key newly generated (<name_of_key>.pub), you can set up the SSH connection. The associated account should have read access rights.
You can follow the official documentation here: GitLab, GitHub, Bitbucket.

c. Configure Sifflet

1. Create the credential that contains the private key that Sifflet uses to connect.

   • In Sifflet, go to Integrations > Credentials

   • Click New credential

   • Information required:

     • Name of the credential

     • The credential format is the following: the exact content of your private key <name_of_key> previously created.

       For GitHub:

       -----BEGIN EC PRIVATE KEY-----
       ########################
       -----END EC PRIVATE KEY-----

       For GitLab/Bitbucket:

       -----BEGIN RSA PRIVATE KEY-----
       ########################
       -----END RSA PRIVATE KEY-----
       

2. Edit the Looker data source you created in part 1 to finalize the connection to your LookML repository:

   • Go to Integrations > Sources, choose your data source, and then click Edit
   • Click on Add Git Configuration

• Information required:
  • URL: the URL of your repo. It should have the following format git@####.git. Choose the SSH option. For instance, for GitLab, you should see the below
  • Secret: the credential you just created
  • Branch (optional): the default branch if left empty. You can also specify the branch you want Sifflet to sync with
  • Auth Type: use SSH

Sifflet Insights Browser Extension

Sifflet Insights browser extension supports the following Looker asset types:

• Dashboards

FAQ

I Cannot Find Some of My Looker Objects on the Catalog/Lineage

If you cannot find some Looker objects in the Sifflet Data Catalog and in the lineage, this could be due to restricted access on some folders. Make sure to give the Sifflet user view access level to all projects you want to be cataloged and part of the lineage. For more information, you can follow the official Looker documentation.