What does Atlan crawl from Databricks?
Atlan crawls and maps the following assets and properties from Databricks.
info
The following properties aren't crawled by the System tables extraction method:
- Table properties:
partitionList,partitionCount - Column properties:
maxLength,precision
Databases
Atlan maps databases from Databricks to its Database asset type.
| Source property | Atlan property | Where in Atlan |
|---|---|---|
TABLE_CATALOG | name | Asset profile and overview sidebar |
SCHEMA_COUNT | schemaCount | API only |
Schemas
Atlan maps schemas from Databricks to its Schema asset type.
| Source property | Atlan property | Where in Atlan |
|---|---|---|
TABLE_SCHEMA | name | Asset profile and overview sidebar |
TABLE_COUNT | tableCount | Asset preview and profile |
VIEW_COUNT | viewsCount | Asset preview and profile |
TABLE_CATALOG | databaseName | Asset preview and profile |
Tables
Atlan maps tables from Databricks to its Table asset type.
| Source property | Atlan property | Where in Atlan |
|---|---|---|
TABLE_NAME | name | Asset profile and overview sidebar |
REMARKS, DESCRIPTION | description | Asset profile and overview sidebar |
COLUMN_COUNT | columnCount | Asset profile and overview sidebar |
LOCATION | externalLocation | Overview sidebar |
FORMAT | externalLocationFormat | Overview sidebar |
OWNER | Created (in Databricks) | Properties sidebar |
CREATEDAT | sourceCreatedAt | Properties sidebar |
UPDATED_BY | Last updated | Properties sidebar |
LASTMODIFIED | sourceUpdatedAt | Properties sidebar |
PARTITIONS | isPartitioned, partitionCount, partitionList | API only |
Views
Atlan maps views from Databricks to its View asset type.
| Source property | Atlan property | Where in Atlan |
|---|---|---|
TABLE_NAME | name | Asset profile and overview sidebar |
REMARKS | description | Asset profile and overview sidebar |
COLUMN_COUNT | columnCount | Asset profile and overview sidebar |
CREATETAB_STMT | definition | Asset profile and overview sidebar |
OWNER | Created (in Databricks) | Properties sidebar |
CREATEDAT | sourceCreatedAt | Properties sidebar |
UPDATED_BY | Last updated | Properties sidebar |
LASTMODIFIED | sourceUpdatedAt | Properties sidebar |
Materialized views
Atlan maps materialized views from Databricks to its MaterialisedView asset type.
| Source property | Atlan property | Where in Atlan |
|---|---|---|
TABLE_NAME | name | Asset profile and overview sidebar |
REMARKS | description | Asset profile and overview sidebar |
COLUMN_COUNT | columnCount | Asset profile and overview sidebar |
CREATETAB_STMT | definition | Asset profile and overview sidebar |
OWNER | Created (in Databricks) | Properties sidebar |
CREATEDAT | sourceCreatedAt | Properties sidebar |
UPDATED_BY | Last updated | Properties sidebar |
LASTMODIFIED | sourceUpdatedAt | Properties sidebar |
Columns
Atlan maps columns from Databricks to its Column asset type.
| Source property | Atlan property | Where in Atlan |
|---|---|---|
PRIMARY KEY | isPrimary | Asset preview and filter, overview sidebar |
FOREIGN KEY | isForeign | Asset preview and filter, overview sidebar |
COLUMN_NAME | name | Asset profile and overview sidebar |
REMARKS | description | Asset profile and overview sidebar |
ORDINAL_POSITION | order | Asset profile |
TYPE_NAME | dataType | Asset profile and overview sidebar |
PARTITION_INDEX | isPartition | Asset preview and profile |
NULLABLE | isNullable | API only |
CHAR_OCTET_LENGTH | maxLength | API only |
DECIMAL_DIGITS | precision | API only |