Lineage & Impact Analysis

How to get a quick preview of your data flow using Atlan's Lineage & Impact Analysis

What is Data Lineage?

A Data Lineage reveals how data has evolved through its lifecycle β€” how and where it has come from. It is a medium to trace back the data sources it is derived from and the transformational steps it has gone through.

The main objective of a good Data Lineage is to make the process of backtracking the data's origins as easy as possible.

Atlan's Data Lineage gives a clear picture of the entire flow of the data and makes it easy for the user to navigate through it.

✨ Spotlight: Atlan offers a fully automated end-to-end Data Lineage for each data asset across multiple sources.

Apart from providing all information about a data asset's source and creation, Atlan also shows other data assets that are dependent on the original asset and how they will be affected by changes β€” all down to a column level, and even including widgets and dashboards from your BI tools.

πŸ‘ How can Data Lineage help you?

  1. πŸ” Lets users independently track a data quality issue back to the source without bugging the engineering team

  2. βœ… Gives quick access to the logic of how a particular column or metric was created

  3. ❗ Foresees the list of data tables or columns that will be affected before making any schema changes

πŸš€ Feature highlights

↔️ Differentiate between source and impacted data assets

Data Lineage gives a clear visual demarcation between the source (or origin) asset and impacted assets. All source assets are shown using green connecting lines, and impacted assets using red connecting lines.

πŸ“‹ Check details about the asset in the preview

You can click any asset in the Lineage diagram to check its details. The preview shows the source of the assets, number of rows and columns, and classification (if any). If you click on the asset name in the preview, you'll be redirected to that asset's lineage.

βš™οΈ View the code in the preview

Just like asset details, you can also preview the code that is transforming the data asset. Just click on the circle and click on Show raw SQL query to view the intermediary code.

πŸ“œ Get SQL Query Summary

Quickly glance over through which assets are impacted by a particular SQL query.

πŸ‘€ Linked assets get highlighted on click

When you click on any particular asset, only the linked assets are highlighted. This helps you focus on the asset you want to learn more about.

πŸ“₯ Download the details of impacted assets

If you want to see how an asset will impact other assets, you can do that right from this screen.

  1. Click on the asset whose impacted assets you need.

  2. Check the preview, and click on Show impacted asset details

  3. A table with all the details of the impacted assets will show up.

  4. Click on the download button, and a file with the names and links of all your impacted assets will get downloaded.

πŸ” Expand the view to full screen

You can expand the Lineage view to full screen and zoom in or out as needed.

➑️ Indirect lineage

Atlan also shows how an asset indirectly impacts other assets. For example, say you are filtering a table by the column profit to show the most profitable products. Here, the column profit is indirectly impacting the entire output table.

πŸ”— Classification propagation

If a classification is attached to an asset, it will flow down to all the secondary assets that are created using the original asset. For example, if the column Customer Name is tagged as PII, then all the columns created using Customer Name will automatically have PII tagged to it. This reduces a lot of manual work and makes sure your data is secure.

πŸ› οΈ How to set up Lineage in Atlan

Your Lineage set-up depends your organization's use case and infrastructure. We’ll work with you and your team to automate the process and help set up Lineage.

Here are some of the sources we support.

1️. SQL queries

By parsing the SQL queries from warehouses like Redshift or Snowflake, Atlan can automatically create all the relationships among the tables and columns. Atlan picks SQL scripts from cloud storage to parse the queries and create lineage.

2️. BI tools

Atlan can use the information provided by your BI tool (like Tableau, Sisense or Redash) to create the Lineage β€” connecting the widgets and dashboards to the source tables and columns that were used to create them.

Tableau assets linked with source Snowflake tables


For sources that Atlan doesn’t support just yet, we won't make you manually create the Lineage for your organization. Atlan's flexible REST APIs can be used to push the relationships to Atlan and see Lineage come to life. This can be very helpful for creating Lineage from ETL tools like Airflow.

Using an easy REST API to push Lineage information to Atlan
RestAPI creates the process entity with inputs and outputs

Seeing how data flows across different sources in your organization can be very powerful and save a lot of time spent on internal communication and debugging.

At Atlan, we understand that end-to-end Lineage can be complex and dependent on your data architecture. We'd be happy to discuss your infrastructure and see how a Lineage can be created for your sources.

4. DIY Lineage

You and your analyst can now create lineage from the Atlan UI itself.

It is super easy and quick to use. Just open lineage tab of any asset without lineage, click on the "Add Lineage" action on the preview panel and start creating.

You can select the origin asset, define the transformation process, and link it to the output asset. And tada, your lineage diagram is ready!

DIY Lineage

πŸ‘€ Note: You can also delete an existing lineage by clicking on the "Delete Process" button.