Steps to integrate your Looker assets with Atlan

Atlan supports integration with Looker, which allows you to integrate your Looker assets ๐Ÿ“Š with your Atlan workspace.

Looker Assets inside Atlan

๐Ÿ’ญ TL;DR

You can set up a Looker integration with your Atlan workspace in four easy steps:

  1. Select the source, aka Looker ๐Ÿ˜‰

  2. Provide your credentials โœ๏ธ

  3. Set up your configuration ๐Ÿ—„๏ธ

  4. Schedule automatic updates ๐Ÿ•‘

๐Ÿ“œ Prerequisites for Looker integration

Before you get started, you'll need some information to help establish a connection between Atlan and your Looker account.

  • Instance URL

  • Client ID and Client Secret

  • Port

๐Ÿ”‘ Looker user permissions

To crawl all the Looker assets a custom role needs to be created having the permissions for Explore and See Sql along with viewer role.

A Viewer role can be used to create the API key to crawl the assets in Atlan, but this will be able to crawl only the widgets and the dashboards. Looker requires special privileges to be given to be able to crawl explores, SQLs of the widgets and create lineage.

Steps to create a custom role

  • Go to Looker instance and open the Admin tab.

  • Select Roles from the drop-down list

  • Click on New Permission Set and give a name to the permission set.

  • Tick the following permissions and save the permission set. Upon saving, you'll be redirected to the Roles page.

  • The following permissions are required to have looks, dashboards and explores crawled in Atlan -

    • access_data

    • see_lookml_dashboards

    • see_looks

    • see_user_dashboards

    • explore (optional, needed if Looker Explores has to be crawled)

  • The following permissions are also required to create lineage between data source and looker assets -

    • see_lookml

    • develop

    • see_datagroups

    • see_sql

Note: Read More regarding Permission Set.

How to choose permission in Looker

โ€‹Permission Set Permission Setโ€‹

  • Click on New Role and give it a name.

  • Select the Permission Set created earlier.

  • Select all the Model Sets you want to gives access to.

  • Select a user to apply the role on and generate the API key.

Roles Example

๐Ÿš€ A step-by-step guide to integrate Looker

Once you have the prerequisite information listed above, follow the steps below ๐Ÿ‘‡ to establish a connection and integrate Atlan with Looker.

STEP 1: Select the source

  1. Log into your Atlan workspace.

  2. Click the "+ New Integration" button on the top right corner of the home screen. You will see a dialogue box with the list of sources available on your workspace.

  3. Select "Looker" from the list of options, and click "Next".

Select Looker integration

STEP 2: Provide credentials

  1. You will get an option to either select "Existing Credentials" (on the left) or "Create a New Credential".

  2. To set up a new connection, click on the "Create Credential" and fill in your Looker credentials. Here are sample values:

    • Instance Name: example.cloud.looker

    • Port: 443

    • Client ID: KJFSH**DSKDH

    • Client Secret: KHFSKH_**_SDSHJ

    • Advanced Config: [optional, needed to create lineage] Check Incoming Data Sources Connection Info to add connection details of the data source (eg. Redshift, Snowflake) connected to LookMLs in Looker.

      There can be multiple connections depend your Looker instance connections.

      • Connection Name: (Name should be same as given in Looker Connection Details) Ex - snowlooker,

      • Data Source: Snowflake, Redshift etc

      • Connection's Hostname : Host URL (Example: clusterName.ap-south-1.redshift.vendor.com)

      • Source Database

      • [Optional] Source Schema

  3. Once you have filled in all the details, click on "Next".

Looker Credentials Setup

STEP 3: Set up the configuration

  1. Select folders you want to crawl.

  2. Choose whether to run the crawler once or schedule it for a daily, weekly, or monthly run. You will be asked to specify the time zone for the run.

  3. Click on "Create". Your connection is now created.

Select folder configuration

Congratulations! Your integration with Looker is now successful ๐ŸŽ‰

๐Ÿ Monitoring your Looker integration

Once the integration setup is completed, you will be redirected to the "Monitor" tab for your Looker asset, where you can monitor the progress of your integration.

๐Ÿ™‹ FAQ

What is each permission used for?

  • access\_data: The topmost permission, which is required to get other permissions like see_look.

  • see\_lookml\_dashboards: Allows Atlan to see and crawl the LookML Dashboard.

  • see\_looks: Allows Atlan to see and crawl Looks.

  • see\_user\_dashboards: Allows Atlan to see and crawl user-defined dashboards.

  • explore: Allows Atlan to see and fetch from the Explore page.

  • see\_lookml: Allows Atlan to see LookML to get model information.

  • develop: Allows Atlan to see connection_name in the model information. This is required for the connection_name mapping with model sets and is used in lineage generation.

  • see\_datagroups: Allows Atlan to see all connection_name and is used in lineage generation.

  • see\_sql: Allows Atlan to fetch the SQL of a query or Look, which is primarily used in lineage generation.

What is connection_name?

The connection_name is similar to Connections you have in the the Looker instance. This is the same terminology as in Looker.

Why wasn't the lineage between the data sources and Looker assets generated?

Here are some troubleshooting steps to check:

  • Check that Advanced Config is present in your credentials.

  • Make sure that Automatically generate lineage for crawled looks and dashboards is checked in the Crawler Config.

  • Check that permission see_sql is given to the role from which API3 credentials are generated.

  • Make sure the data sources' assets are in Atlan.

What tiles (elements) are crawled from the Looker Dashboard?

Two tiles are crawled from Looker:

  • Look tile

  • Query tile

If the Explore asset isn't crawled, will the lineage between the data sources and Looker assets be generated?

Yes, the table lineage will still be formed with Widget.