What does Atlan crawl from Dremio

||Connect docs via MCP

Atlan crawls comprehensive metadata from your Dremio data lakehouse, including physical datasets, virtual datasets, folders, spaces, and sources information.

Assets

Atlan crawls the following Dremio assets, each with specific metadata fields.

DremioSpace

Dremio Spaces represent logical workspaces where users create and organize virtual datasets.

Source field	Atlan field	Description
name	name	Space name
description	description	Space description
tags	dremioLabels	Labels/tags associated with the space
id	dremioId	Unique identifier for the space in Dremio

DremioSource

Dremio Sources represent external data connections providing access to various systems.

Source field	Atlan field	Description
name	name	Source name
description	description	Source description
tags	dremioLabels	Labels/tags associated with the source
type	dremioSourceType	Type of external source connection
config	dremioSourceConnectionConfigs	Configuration parameters for connecting to the external source
accelerationGracePeriodMs, accelerationRefreshPeriodMs, accelerationRefreshSchedule, accelerationActivePolicyType, accelerationNeverExpire, accelerationNeverRefresh	dremioSourceAccelerationSettings	Acceleration settings including grace period, refresh settings, and policy configuration
metadataPolicy	dremioSourceMetadataPolicies	Metadata policy settings for the source
id	dremioId	Unique identifier for the source in Dremio

DremioFolder

Dremio Folders represent organizational containers within spaces or sources.

Source field	Atlan field	Description
name	name	Folder name
description	description	Folder description
tags	dremioLabels	Labels/tags associated with the folder
id	dremioId	Unique identifier for the folder in Dremio

Physical Dataset

Physical datasets represent tables from connected data sources available in Dremio.

Source field	Atlan field	Description
name	name	Physical dataset name
description	description	Physical dataset description
tags	dremioLabels	Labels/tags associated with the dataset
rowCount	rowCount	Number of rows in the dataset
columnCount	columnCount	Number of columns in the dataset
createdAt	sourceCreatedAt	When the physical dataset was created
id	dremioId	Unique identifier for the dataset in Dremio
owner	sourceOwners	Owner of the dataset

Virtual Dataset

Virtual datasets represent virtual tables created in Dremio through data virtualization.

Source field	Atlan field	Description
name	name	Virtual dataset name
sql	definition	SQL definition of the virtual dataset
description	description	Virtual dataset description
tags	dremioLabels	Labels/tags associated with the dataset
createdAt	sourceCreatedAt	When the virtual dataset was created
id	dremioId	Unique identifier for the dataset in Dremio
rowCount	rowCount	Number of rows in the dataset
columnCount	columnCount	Number of columns in the dataset
owner	sourceOwners	Owner of the dataset

Column

Columns represent individual fields within physical and virtual datasets.

Source field	Atlan field	Description
name	name	Column name
type	dataType	Column data type
isNullable	isNullable	Whether the column permits null values
description	description	Column description
tags	dremioLabels	Labels/tags associated with the column
id	dremioId	Unique identifier for the column in Dremio
isPrimaryKey	isPrimaryKey	Whether the column is a primary key
isPartitionColumn	isPartitionColumn	Whether the column is a partition column
ordinalPosition	ordinalPosition	Position of the column in the table
precision	precision	Precision of the column
scale	scale	Scale of the column
owner	sourceOwners	Owner of the column
createdAt	sourceCreatedAt	When the column was created

Lineage

Atlan builds lineage for Dremio by parsing each virtual dataset's SQL definition:

• Virtual-to-physical (intra-Dremio): each virtual dataset is linked to the physical datasets—and upstream virtual datasets—it selects from, including multi-hop virtual-to-virtual chains.
• Cross-connector upstream: a Dremio physical dataset is linked to the real source table already in Atlan—AWS Glue and Iceberg tables (lakehouse) and relational tables such as Snowflake, PostgreSQL, or Redshift—when you select the source connection under Advanced Configuration during crawler setup and that source is crawled in Atlan.

Requirements:

• The crawling role needs SELECT on the datasets whose SQL is parsed (granted during setup—reading each view's SQL definition from INFORMATION_SCHEMA.VIEWS).
• For cross-connector upstream lineage, the upstream source connection must already exist in Atlan, and you must select it under Advanced Configuration.