Skip to main content

Connection issues

This guide helps you resolve common connection and authentication issues when setting up the dbt connector in Atlan.

What are the known limitations of dbt connector?​

Following are the known limitations:

  • For dbt Core, Atlan currently only processes the status of dbt tests - passed, failed, error and warning - but not the job name and execution time. However, this information is available for dbt Cloud. You can find this information in asset sidebar.
  • Column-level lineage for dbt models is currently unavailable at source. However, Atlan applies a custom SQL-parsing algorithm to the materialized SQL tables to generate column-level lineage for dbt models. Note that due to the limitations of SQL parsing, Atlan doesn't guarantee generating lineage for all columns.
  • Ephemeral dbt models don't materialize tables at source. Hence, these doesn't create lineage for the materialized layer or publish dbt model columns in Atlan. This is an expected behavior of ephemeral dbt models.
  • In case you are using Dbt Cloud, and your assets gets executed by multiple jobs of different environment, Atlan currently takes the last run accross all environment and update the underlying asset metadata and lineage. This means that the materialized asset are also impacted.
Please note that this is no longer applicable in the revamped experience. Each environment run is presented with a separate asset in Atlan, which means each asset has the information of the environment where it was executed only, and by consequence each asset and its materialized assets have the right metadata, lineage and run status. For more information please refer to Dbt Reavmped Experience.
  • A dbt source can include multiple SQL warehouse tables. Lineage between dbt sources and models isn't supported. Instead, Atlan builds lineage between dbt models and SQL tables when dbt models have an explicit dependency on specific SQL tables of a dbt source.
Please note that this is no longer applicable in the revamped experience. Each dbt source table is treated as a separate asset. Atlan recommend upgrading to benefit from the improved capabilities. For more information please refer to Dbt Reavmped Experience.

Does Atlan support certifiying assets, changing descriptions, owners, managing announcement, tagging assets, linking terms, readmes and Atlan domains and updating custom metadata from dbt?​

Yes, you can do all these enrichmenents from dbt to Atlan. For more details, refer to the Enrich Atlan through dbt

Why are my dbt Cloud models not showing up in Atlan after setup?​

Atlan only crawls dbt assets that are in the “applied” (built) state in dbt Cloud. Models must be part of a successful run to be picked up during crawling; models that are only defined in your project files but haven’t been executed won’t be included. For more information about project state, see Project states in dbt Cloud.

Why are columns for dbt models missing?​

If you've crawled your dbt models but columns are missing:

  • Define the columns in the columns attribute of the model's .yml definition in the manifest.json file.
  • Check if the dbt model is materializing a table or view asset, without which columns for a dbt model don't show up.

Does Atlan support data types for dbt model columns?​

Yes, Atlan supports column data types for dbt models:

  • dbt Cloud: Make sure there is at least one job run with the Generate docs on run option enabled for every environment where dbt models are executed.
  • dbt Core - upload the manifest.json and catalog.json files generated by the dbt docs generate command for every dbt Core project. Refer to dbt Core documentation to learn how to structure the bucket while uploading your files.

Is dbt source metadata available for materialized columns?​

If metadata enrichment is enabled for assets that dbt materializes, only table assets are updated with the materialized by dbt source or model or seed metadata. Materialized columns are only updated with dbt model or seeds metadata - dbt source metadata is currently not supported for column assets.

Please note that this is no longer applicable in the revamped experience. Dbt source metadata is applied just like dbt models and seeds on materialized columns. For more information please refer to Dbt Reavmped Experience.

Can I map Atlan GitHub action to multiple dbt projects?​

Yes, you can configure the Atlan GitHub action for multiple dbt projects.

Why are some dbt tests missing?​

Atlan doesn't support crawling dbt tests with an auto generated unique_id that exceeds the character limit of 32,000 characters. If you want to catalog such dbt tests in Atlan, you need to define a custom name for your dbt tests within the character limit.

Why is there a discrepancy in dbt test count between dbt and Atlan?​

Atlan fetches dbt models, sources, and tests from the applied state of each dbt environment. This is the most recent project state for each environment. The asset count on Atlan may differ from what’s present in the manifest files for individual job runs. Refer to dbt documentation to learn more about dbt project states.

If a materialized table is linked to multiple dbt assets, Atlan applies the description from the linked dbt model to the materialized asset. In case this is unavailable, Atlan then applies the description from the linked dbt source to the asset.

If a materialized table is linked to multiple dbt assets, Atlan applies the description from the linked dbt seed to the materialized asset. In case this is unavailable, there is no description available.

Was this page helpful?