What does Atlan crawl from Databricks?

✦TL;DR

During a Databricks crawl, Atlan extracts metadata for Databases, Schemas, Tables, Views, Materialized Views, Volumes, External Locations, and AI Models. Reference tables map each Databricks property to its Atlan asset.

||

Your AI can read this via Docs MCPcurl -fsSL "https://docs.atlan.com/install-docs-mcp" | bashConnect

Atlan crawls Databricks metadata for databases, schemas, tables, views, materialized views, volumes, external locations, and AI models, mapping each to its Atlan asset. The tables below list which Databricks properties are read and how they appear on the corresponding Atlan asset.

Lineage

Atlan supports complete lineage tracking for the following Databricks assets:

• Tables
• Views
• Materialized views
• Metric views
• Volumes
• External locations
• AI model versions (requires additional permissions)
• AI/BI dashboards (private preview, requires additional permissions)
• Genie Agents (private preview, requires additional permissions)

Unity Catalog

Unity Catalog is Databricks' governance layer for data and AI. For connection scope, cross-workspace behavior, and which governance objects aren't imported into Atlan, see Unity Catalog governance and connections.

Tags

Atlan imports Unity Catalog tags and resolves which assets carry them. Tags are matched case-insensitively to Atlan tags; when a match is found, the tag and its value are applied to every Databricks asset that carries it in Unity Catalog. Reverse sync is supported: attaching, detaching, or updating a tag value in Atlan pushes the change back to Databricks.

Source property	Atlan property	Where in Atlan
Unity Catalog tag name	Matched Atlan tag	Asset profile and overview sidebar
Tag allowed values (governed tags)	tagAllowedValues	Tag manager
Tag value assigned to asset	Atlan tag value on the asset	Asset profile and overview sidebar

Governed tags

When a Unity Catalog tag is registered as a Tag Policy with a list of allowed values, Atlan surfaces those values as a dropdown selector in the tag editor rather than a free-text field.

Assets

Atlan crawls and maps the following assets and properties from Databricks.

info

The following properties aren't crawled by the System tables extraction method:

• Table properties: partitionList, partitionCount
• Column properties: maxLength, precision

Databases

Atlan maps databases from Databricks to its Database asset type.

Source property	Atlan property	Where in Atlan
TABLE_CATALOG	name	Asset profile and overview sidebar
SCHEMA_COUNT	schemaCount	API only

Schemas

Atlan maps schemas from Databricks to its Schema asset type.

Source property	Atlan property	Where in Atlan
TABLE_SCHEMA	name	Asset profile and overview sidebar
TABLE_COUNT	tableCount	Asset preview and profile
VIEW_COUNT	viewsCount	Asset preview and profile
TABLE_CATALOG	databaseName	Asset preview and profile

Tables

Atlan maps tables from Databricks to its Table asset type.

Source property	Atlan property	Where in Atlan
TABLE_NAME	name	Asset profile and overview sidebar
REMARKS, DESCRIPTION	description	Asset profile and overview sidebar
COLUMN_COUNT	columnCount	Asset profile and overview sidebar
LOCATION	externalLocation	Overview sidebar
FORMAT	externalLocationFormat	Overview sidebar
OWNER	Created (in Databricks)	Properties sidebar
CREATEDAT	sourceCreatedAt	Properties sidebar
UPDATED_BY	Last updated	Properties sidebar
LASTMODIFIED	sourceUpdatedAt	Properties sidebar
PARTITIONS	isPartitioned, partitionCount, partitionList	API only

Views

Atlan maps views from Databricks to its View asset type.

Source property	Atlan property	Where in Atlan
TABLE_NAME	name	Asset profile and overview sidebar
REMARKS	description	Asset profile and overview sidebar
COLUMN_COUNT	columnCount	Asset profile and overview sidebar
CREATETAB_STMT	definition	Asset profile and overview sidebar
OWNER	Created (in Databricks)	Properties sidebar
CREATEDAT	sourceCreatedAt	Properties sidebar
UPDATED_BY	Last updated	Properties sidebar
LASTMODIFIED	sourceUpdatedAt	Properties sidebar

Metric views

Atlan maps metric views from Databricks to its DatabricksMetricView asset type.

Note

upstream column lineage for dimensions/measures isn't supported.

Source property	Atlan property	Where in Atlan
TABLE_NAME	name	Asset profile and overview sidebar
REMARKS	description	Asset profile and overview sidebar
COLUMN_COUNT	columnCount	Asset profile and overview sidebar
CREATETAB_STMT	definition	Asset profile and overview sidebar
OWNER	Created (in Databricks)	Properties sidebar
CREATEDAT	sourceCreatedAt	Properties sidebar
UPDATED_BY	Last updated	Properties sidebar
LASTMODIFIED	sourceUpdatedAt	Properties sidebar

Materialized views

Atlan maps materialized views from Databricks to its MaterialisedView asset type.

Source property	Atlan property	Where in Atlan
TABLE_NAME	name	Asset profile and overview sidebar
REMARKS	description	Asset profile and overview sidebar
COLUMN_COUNT	columnCount	Asset profile and overview sidebar
CREATETAB_STMT	definition	Asset profile and overview sidebar
OWNER	Created (in Databricks)	Properties sidebar
CREATEDAT	sourceCreatedAt	Properties sidebar
UPDATED_BY	Last updated	Properties sidebar
LASTMODIFIED	sourceUpdatedAt	Properties sidebar

External location

Atlan maps external locations from Databricks to its DatabricksExternalLocation asset type.

Source property	Atlan property	Where in Atlan
EXTERNAL_LOCATION_NAME	name	Asset profile and overview sidebar
COMMENT	description	Asset profile and overview sidebar
EXTERNAL_LOCATION_OWNER	databricksOwner	Properties sidebar
CREATED	sourceCreatedAt	Properties sidebar
CREATED_BY	sourceCreatedBy	Properties sidebar
LAST_ALTERED	sourceUpdatedAt	Properties sidebar
LAST_ALTERED_BY	sourceUpdatedBy	Properties sidebar
URL	databricksUrl	Overview sidebar

Volumes

Atlan maps volumes from Databricks to its Volume asset type.

Source property	Atlan property	Where in Atlan
VOLUME_NAME	name	Asset profile and overview sidebar
COMMENT	description	Asset profile and overview sidebar
VOLUME_TYPE	databricksVolumeType	Asset profile and overview sidebar
VOLUME_OWNER	databricksVolumeOwner	Properties sidebar
CREATED	sourceCreatedAt	Properties sidebar
CREATED_BY	sourceCreatedBy	Properties sidebar
LAST_ALTERED	sourceUpdatedAt	Properties sidebar
LAST_ALTERED_BY	sourceUpdatedBy	Properties sidebar
STORAGE_LOCATION	databricksVolumeExternalLocation	Overview sidebar
VOLUME_CATALOG	databaseName	Asset preview and profile
VOLUME_SCHEMA	schemaName	Asset preview and profile

Columns

Did you know?

To help you work seamlessly with STRUCT data types, Atlan supports nested columns up to level 30 in Databricks. You can view these columns in the Tree view or the asset sidebar of your table assets, and also explore child columns of STRUCTs nested within MAPs or ARRAYs. However, lineage for nested columns isn't supported.

Atlan maps columns from Databricks to its Column asset type.

Source property	Atlan property	Where in Atlan
PRIMARY KEY	isPrimary	Asset preview and filter, overview sidebar
FOREIGN KEY	isForeign	Asset preview and filter, overview sidebar
COLUMN_NAME	name	Asset profile and overview sidebar
REMARKS	description	Asset profile and overview sidebar
ORDINAL_POSITION	order	Asset profile
TYPE_NAME	dataType	Asset profile and overview sidebar
PARTITION_INDEX	isPartition	Asset preview and profile
NULLABLE	isNullable	API only
CHAR_OCTET_LENGTH	maxLength	API only
DECIMAL_DIGITS	precision	API only

AI models

AI models are crawled only when additional permissions are granted and the Direct extraction strategy is used. Only models registered in the Unity Catalog Model Registry are crawled.

AI models

Atlan maps AI models from Databricks to its AIModel asset type.

Source property	Atlan property	Where in Atlan
Model name	name	Asset profile and overview sidebar
COMMENT	description	Asset profile and overview sidebar
Catalog	databaseName	Asset preview and profile
Schema	schemaName	Asset preview and profile
Owner	sourceOwner	Properties sidebar
Created at	sourceCreatedAt	Properties sidebar

AI model versions

Atlan maps AI model versions from Databricks to its AIModelVersion asset type.

Source property	Atlan property	Where in Atlan
Version	modelVersionName	Asset profile and overview sidebar
Version status	modelVersionStatus	Asset profile and overview sidebar
Source URI	modelSourceURL	Overview sidebar
Artifact URI	modelArtifactPath	Overview sidebar
Aliases	modelVersionAliases	Overview sidebar
Dataset count	modelDatasetCount	API only
Run ID	modelRunId	Properties sidebar
Run name	modelRunName	Properties sidebar
Run start time	modelRunStartTime	Properties sidebar
Run end time	modelRunEndTime	Properties sidebar
Metrics	modelMetrics	API only
Parameters	modelParameters	API only

AI/BI dashboards

Dashboard metadata comes from the Databricks Lakeview API and requires the Import AI/BI Dashboards option. See Crawl Databricks AI/BI dashboards.

Source property	Atlan property	Where in Atlan
display_name	name	Asset profile and overview sidebar
path	databricksDashboardPath	Overview sidebar
warehouse_id	databricksDashboardWarehouseId	Overview sidebar
lifecycle_state	databricksDashboardLifecycleState	Properties sidebar
etag	databricksDashboardEtag	API only
dashboard_id	sourceId	Properties sidebar
Workspace identifier	databricksDashboardWorkspaceId	Properties sidebar
Genie space association	databricksDashboardIsGenieSpaceEnabled	Properties sidebar
Published dashboard link	sourceURL	Asset profile
CAN MANAGE user grants	ownerUsers	Overview sidebar

Genie spaces

Genie metadata comes from the Databricks Genie API and requires the Import Genie Agents option. Atlan maps each Genie space to its DatabricksGenieAgent asset type. See Crawl Databricks Genie Agents.

Source property	Atlan property	Where in Atlan
title	name	Asset profile and overview sidebar
space_id	assetSourceId	Properties sidebar
description	description	Asset profile and overview sidebar
Workspace identifier	databricksGenieAgentWorkspaceId	Properties sidebar
warehouse_id	databricksGenieAgentWarehouseId	Properties sidebar
parent_path	databricksGenieAgentParentPath	Properties sidebar
etag	databricksGenieAgentEtag	API only