What does Atlan crawl from dbt Core?
Atlan crawls and maps the following assets and properties from dbt Core. Atlan also supports lineage between the following:
- dbt models
- dbt seeds
- dbt sources
- SQL tables and views materialized by dbt models, dbt seeds, dbt sources
- Column-level lineage for these entities
Once you've crawled dbt, you can use dbt-specific filters for quick asset discovery:
Test status- filter dbt tests that passed, failed, or have a warning or errorAlias- filter by the name of a dbt model's identifier in the dbt projectUnique id- filter by the unique node identifier of a dbt modelProject name- filter by dbt project name, only supported for dbt Core version 1.6+Job name- filter by dbt job nameEnvironment name- filter by dbt environment nameJob status- filter by dbt job statusLast job run- filter by the last run of the dbt job
Atlan's dbt crawler also populates custom metadata to further enrich the assets in Atlan. The Atlan dbt-specific property column in the tables below gives the name of the mapped custom metadata property in Atlan.
Atlan enables you to sync your dbt tags and update your dbt assets with the synced tags. You can also map other metadata to Atlan's assets through your dbt models.
Tables and views
Atlan maps materialized tables and views from dbt Core to its Table/View asset type.
| Source property | Atlan property | Where in Atlan |
|---|---|---|
description | description | asset profile and overview sidebar |
alias | assetDbtAlias | asset filter and properties sidebar |
uniqueId | assetDbtUniqueId | asset filter and overview sidebar |
accountName | assetDbtAccountName | asset qualified name |
projectName | assetDbtProjectName | asset filter and overview sidebar |
packageName | assetDbtPackageName | asset filter and properties sidebar |
tags | assetDbtTags | asset filter and overview sidebar |
meta | assetDbtMeta | API only |
| [collected via dbt files] | assetDbtTestStatus | API and dbt test sidebar |
Columns
Atlan maps materialized columns from dbt Core to its Column asset type.
| Source property | Atlan property | Where in Atlan |
|---|---|---|
description | description | asset profile and overview sidebar |
alias | assetDbtAlias | asset filter and properties sidebar |
uniqueId | assetDbtUniqueId | asset filter and overview sidebar |
accountName | assetDbtAccountName | asset qualified name |
projectName | assetDbtProjectName | asset filter and overview sidebar |
packageName | assetDbtPackageName | properties sidebar |
tags | assetDbtTags | asset filter and overview sidebar |
meta | assetDbtMeta | API only |
| [collected via dbt files] | assetDbtTestStatus | dbt test sidebar |
Models
Atlan maps models from dbt Core to its DbtModel asset type.
| Source property | Atlan property | Where in Atlan |
|---|---|---|
name | name | asset profile and overview sidebar |
description | description | asset profile and overview sidebar |
owner | sourceCreatedBy | asset profile and properties sidebar |
alias | assetDbtAlias | asset filter and properties sidebar |
uniqueId | assetDbtUniqueId | asset filter and overview sidebar |
accountName | assetDbtAccountName | asset qualified name |
projectName | assetDbtProjectName | asset filter and overview sidebar |
packageName | assetDbtPackageName | properties sidebar |
raw_sql or raw_code | dbtRawSQL | overview sidebar |
compiled_sql or compiled_code | dbtCompiledSQL | overview sidebar |
status | dbtJobRuns.dbtModelRunStatus | overview sidebar |
tags | assetDbtTags | asset filter and overview sidebar |
meta | assetDbtMeta | API only |
config.materialized | dbtMaterializationType | API only |
executeCompletedAt | sourceUpdatedAt | asset profile and properties sidebar |
stats | dbtStats | API only |
| [dynamically generated using accountId, projectId, and uniqueId] | sourceURL | overview sidebar |
Sources
Atlan maps sources from dbt Core to its DbtSource asset type.
| Source property | Atlan property | Where in Atlan |
|---|---|---|
name | name | asset profile and overview sidebar |
description | description | asset profile and overview sidebar |
owner | sourceCreatedBy | asset profile and properties sidebar |
alias | assetDbtAlias | asset filter and properties sidebar |
uniqueId | assetDbtUniqueId | asset filter and overview sidebar |
accountName | assetDbtAccountName | asset qualified name |
projectName | assetDbtProjectName | Asset filter and overview sidebar |
packageName | assetDbtPackageName | properties sidebar |
tags | assetDbtTags | asset filter and overview sidebar |
meta | assetDbtMeta | API only |
stats | dbtStats | API only |
freshness | assetDbtSourceFreshnessCriteria | Overview sidebar |
criteria | assetDbtSourceFreshnessCriteria | overview sidebar |
| [dynamically generated using accountId, projectId, and uniqueId] | sourceURL | Overview sidebar |
Tests
For dbt Core, upload the run_results.json file to crawl dbt tests. It's recommended to place the file in the same folder as the manifest.json file.
Atlan maps tests from dbt Core to its DbtTest asset type.
| Source property | Atlan property | Where in Atlan |
|---|---|---|
name | name | asset profile and overview sidebar |
description | description | asset profile and overview sidebar |
alias | assetDbtAlias | asset filter and properties sidebar |
uniqueId | assetDbtUniqueId | asset filter and overview sidebar |
accountName | assetDbtAccountName | asset qualified name |
projectName | assetDbtProjectName | asset filter, overview and properties sidebar |
packageName | assetDbtPackageName | properties sidebar |
raw_code | dbtTestRawCode | overview sidebar |
raw_sql | dbtTestRawSQL | overview sidebar |
compiled_code | dbtTestCompiledCode | overview sidebar |
compiled_sql | dbtTestCompiledSQL | overview sidebar |
status | dbtTestStatus | overview sidebar |
error | dbtTestError | asset profile and overview sidebar |
tags | assetDbtTags | asset filter and overview sidebar |
meta | assetDbtMeta | API only |
| [dynamically generated using accountId, projectId, and uniqueId] | sourceURL | overview sidebar |
Seeds
Atlan maps models from dbt Core to its DbtSeed asset type.
| Source property | Atlan property | Where in Atlan |
|---|---|---|
name | name | asset profile and overview sidebar |
description | description | asset profile and overview sidebar |
executeCompletedAt | sourceUpdatedAt | asset profile and properties sidebar |
owner | sourceCreatedBy | asset profile and properties sidebar |
status | dbtJobRuns.dbtModelRunStatus | overview sidebar |
alias | assetDbtAlias | asset filter and properties sidebar |
meta | assetDbtMeta | API only |
uniqueId | assetDbtUniqueId | asset filter and overview sidebar |
stats | dbtSeedStats | API only |
filePath | dbtSeedfilePath | asset profile and overview sidebar |