Skip to main content

Connection issues

This guide helps you resolve common connection and authentication issues when setting up the dbt connector in Atlan.

What are the known limitations of the dbt connector?​

Following are the known limitations:

  • For dbt Core, Atlan currently only processes the status of dbt tests - passed, failed, and warning - but not the job name and execution time. However, this information is available for dbt Cloud. You can hover over the test Status in the asset sidebar to view this information.
  • Column-level lineage for dbt models is currently unavailable at source. However, Atlan applies a custom SQL-parsing algorithm to the materialized SQL tables to generate column-level lineage for dbt models. Note that due to the limitations of SQL parsing, Atlan doesn't guarantee generating lineage for all columns.
  • Ephemeral dbt models don't materialize tables at source. Hence, these doesn't create lineage for the materialized layer or publish dbt model columns in Atlan. This is an expected behavior of ephemeral dbt models.
  • A dbt source can include multiple SQL warehouse tables. Lineage between dbt sources and models isn't supported. Instead, Atlan builds lineage between dbt models and SQL tables when dbt models have an explicit dependency on specific SQL tables of a dbt source.

Why are my dbt Cloud models not showing up in Atlan after setup?​

Atlan only crawls dbt assets that are in the “applied” (built) state in dbt Cloud. Models must be part of a successful run to be picked up during crawling; models that are only defined in your project files but haven’t been executed won’t be included. For more information about project state, see Project states in dbt Cloud.

Does Atlan support data types for dbt model columns?​

Yes, Atlan supports column data types for dbt models:

  • dbt Cloud: Make sure there is at least one job run with the Generate docs on run option enabled for every environment where dbt models are executed.
  • dbt Core - upload the manifest.json and catalog.json files generated by the dbt docs generate command for every dbt Core project. Refer to dbt Core documentation to learn how to structure the bucket while uploading your files.

Why are columns for dbt models missing?​

If you've crawled your dbt models but columns are missing:

  • Define the columns in the columns attribute of the model's .yml definition in the manifest.json file.
  • Check if the dbt model is materializing a table or view asset, without which columns for a dbt model don't show up.

Is dbt source metadata available for materialized columns?​

If metadata enrichment is enabled for assets that dbt materializes, only table assets are updated with the materialized by dbt source or model metadata. Materialized columns are only updated with dbt model metadata - dbt source metadata is currently not supported for column assets.

Can I map the Atlan GitHub action to multiple dbt projects?​

Yes, you can configure the Atlan GitHub action for multiple dbt projects.

Does Atlan support syncing terms from dbt?​

Yes, you can update terms from dbt to Atlan. For more details, refer to the developer documentation.

Why are some dbt tests missing?​

Atlan doesn't support crawling dbt tests with an auto-generated unique_id that exceeds the character limit of 32,000 characters. If you want to catalog such dbt tests in Atlan, you need to define a custom name for your dbt tests within the character limit.

Why is there a discrepancy in dbt test count between dbt and Atlan?​

Atlan fetches dbt models, sources, and tests from the applied state of each dbt environment. This is the most recent project state for each environment. The asset count on Atlan may differ from what’s present in the manifest files for individual job runs. Refer to dbt documentation to learn more about dbt project states.

If a materialized table is linked to multiple dbt assets, Atlan applies the description from the linked dbt model to the materialized asset. In case this is unavailable, Atlan then applies the description from the linked dbt source to the asset.

If a materialized table is linked to multiple dbt assets, Atlan applies the description from the linked dbt seed to the materialized asset. In case this is unavailable, there is no description available.