What does Atlan crawl from AWS Glue?

Atlan extracts metadata for Catalogs, Databases, Schemas, Tables, and Jobs from AWS Glue and maps each property to its internal asset model. The tables below list which AWS Glue properties are read and how they appear on the corresponding Atlan asset.

Databases

Atlan maps databases from AWS Glue to its Database asset type.

Source property	Atlan property
TABLE_CATALOG	name
SCHEMA_COUNT	schemaCount

Schemas

Atlan maps schemas from AWS Glue to its Schema asset type.

Source property	Atlan property
TABLE_SCHEMA	name
TABLE_COUNT	tableCount
VIEW_COUNT	viewsCount
TABLE_CATALOG	databaseName
COMMENTS	description
CreateTime	sourceCreatedAt

Tables

Atlan maps tables from AWS Glue to its Table asset type.

Source property	Atlan property
TABLE_NAME	name
COMMENTS	description
COLUMN_COUNT	columnCount
ROW_COUNT	rowCount
objectCount	tableObjectCount
BYTES	sizeBytes
Parameters (recordCount)	rowCount
Parameters (sizeKey)	sizeBytes
TABLE_TYPE	subType
StorageDescriptor (Location)	externalLocation
Parameters (typeOfData, classification)	externalLocationFormat
PartitionKeys	isPartitioned
PartitionData	partitionCount, partitionList
CreatedBy	sourceCreatedBy
CreateTime, CreationTime	sourceCreatedAt
UpdateTime, LastAccessTime	sourceUpdatedAt

Views

Atlan maps views from AWS Glue to its View asset type.

Source property	Atlan property
TABLE_NAME	name
COMMENTS	description
COLUMN_COUNT	columnCount
VIEW_DEFINITION	definition
Parameters (recordCount)	rowCount
PartitionKeys	isPartitioned
PartitionData	partitionCount, partitionList
CreatedBy	sourceCreatedBy
CreateTime, CreationTime	sourceCreatedAt
UpdateTime, LastAccessTime	sourceUpdatedAt

Columns

Atlan maps columns from AWS Glue to its Column asset type.

Atlan also supports nested columns up to level 15 for AWS Glue to help you enrich your semi-structured data types:

• Atlan retrieves raw STRUCT and ARRAY type objects for nested columns up to 15 levels.
• View nested columns in the column preview and overview sidebar for your table assets.
• Column-level lineage is supported. Search, enrich metadata, and view lineage for nested columns.
• Tag propagation is currently only supported from parent to nested columns.
• Atlan currently doesn't parse MAP type objects for columns and nested columns.

Source property	Atlan property
COLUMN_NAME	name
COMMENTS	description
ORDINAL_POSITION, COLUMN_ID	order
TYPE_NAME, DATA_TYPE	dataType
IS_PARTITION	isPartition
PARTITION_ORDER	partitionOrder