Add impact analysis in GitLab

warning

For existing users, the dbt-action is no longer maintained and is deprecated eventually. Atlan strongly recommends migrating to the atlan-action. Refer to How to migrate from dbt to Atlan action to learn more and complete the migration.

If you have ever changed a dbt model only to find out later that it broke a downstream table or dashboard, Atlan provides a GitLab CI/CD pipeline to help you out.

This pipeline places Atlan's impact analysis right into your merge request. So, you can view the potential downstream impact of your changes before merging the request.

Prerequisites

1. Before running the action, you need to create an Atlan API token or configure an OAuth client.
2. Assign a persona to the API token and add a metadata policy that provides requisite permissions on assets for the Atlan dbt action to work. For example, you can add the following permissions:
   • dbt - Read and Update
   • Materialized layer, such as Snowflake - Read and Update
   • Any downstream connections, such as Microsoft Power BI - Read only
3. When a merge request with changes to one or more dbt models is merged, the Atlan dbt action links the merge request as a resource to the assets in Atlan. To make sure that the merge request is linked as a resource, you need to assign the right persona with requisite permissions to the API token.

Configure the action

To set up the Atlan dbt action in GitLab:

1. Define CI/CD variables in your repository:
   • ATLAN_INSTANCE_URL with the URL of your Atlan instance.
   • ATLAN_API_TOKEN with the value of the API token.
   • GITLAB_TOKEN with the value of the project access token.
2. Click the checkboxes for Mask variable and Expand variable only. Leave the Protect variable checkbox unchecked - merge request pipelines do not have access to protected variables.
3. Add the GitLab pipeline to your workflow:

   1. Create a workflow file in the root directory of your repository - .gitlab-ci.yml.

   2. Add the following code to your workflow file:

      stages:
        - get-downstream-impact
      
      get-downstream-impact-open:
        stage: get-downstream-impact
        image: node:20
        script:
          - git clone   - branch v1 https://github.com/atlanhq/atlan-action.git
          - cd atlan-action
          - npm install
          - npm run build
          - node ./adapters/index.js
        environment:
          name: get-downstream-impact
        rules:
          - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
          - if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
            when: never
          - if: '$CI_COMMIT_BRANCH'

Test the action

After you've completed the configuration earlier, create a merge request with a changed dbt model file to test the action. You can see the Atlan GitLab CI/CD pipeline running and adding comments in your merge request:

• The GitLab CI/CD pipeline adds and updates a single comment for every file change.
• The impacted assets in the comment are displayed in a collapsible section and grouped by source and asset type.
• The comment includes some metadata for your impacted assets - such as descriptions, owners, and linked glossary terms.
• View the impacted assets in Atlan or open the source URL - for example, view an impacted Looker dashboard directly in Looker.
• Once you have merged the merge request, it's added as a resource to the dbt model and its materialized assets. You can view the linked merge request from the Resources tab of the asset sidebar. For example:

Inputs

Name	Description	Required
GITLAB_TOKEN	For writing comments on PRs to print downstream assets	true
ATLAN_INSTANCE_URL	For making API requests to the user's tenant	true
ATLAN_API_TOKEN	For authenticating API requests to the user's tenant	true
DBT_ENVIRONMENT_BRANCH_MAP	For mapping the GitLab branch with a specific dbt environment	false
IGNORE_MODEL_ALIAS_MATCHING	For turning off matching aliases using this variable	false

Troubleshooting the action

Why does the action fetch a model from an incorrect environment?

If there are multiple dbt models with the same name but across different environments in your Atlan instance, the action may fetch an incorrect model. In order to make sure that the action fetches a model from the right environment, you can map the GitLab branch with a specific dbt environment. This enables the Atlan GitLab CI/CD pipeline to parse lineage for that specific environment.

For example, you can provide the mapping in this format - branch name : dbt environment name

stages:
  - get-downstream-impact

get-downstream-impact-open:
  stage: get-downstream-impact
  image: node:20
  variables:
+    DBT_ENVIRONMENT_BRANCH_MAP: |
+      main: [Enter Your Branch name]
  script:
    - git clone   - branch v1 https://github.com/atlanhq/atlan-action.git
    - cd atlan-action
    - npm install
    - npm run build
    - node ./adapters/index.js
  environment:
    name: get-downstream-impact
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
      when: never
    - if: '$CI_COMMIT_BRANCH'

Why does the action fetch a model by its alternate name and not model name?

By default, the action checks if there is an alternate name defined for a dbt model in the code and looks for the relevant asset in Atlan using that alternate name. To turn off matching alternate names for your dbt models, you can set the IGNORE_MODEL_ALIAS_MATCHING input to true.

For example:

stages:
  - get-downstream-impact

get-downstream-impact-open:
  stage: get-downstream-impact
  image: node:20
  variables:
+    IGNORE_MODEL_ALIAS_MATCHING: "true"
  script:
    - git clone   - branch v1 https://github.com/atlanhq/atlan-action.git
    - cd atlan-action
    - npm install
    - npm run build
    - node ./adapters/index.js
  environment:
    name: get-downstream-impact
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
      when: never
    - if: '$CI_COMMIT_BRANCH'