What does Atlan crawl from Hive?
During a Hive crawl, Atlan extracts metadata for Databases, Schemas, Tables, Views, Materialized Views, and Columns. Reference tables map each Hive property to its Atlan asset.
Atlan extracts metadata for Databases, Schemas, Tables, Views, Materialized Views, and Columns from Hive and maps each property to its internal asset model. The tables below list which Hive properties are read and how they appear on the corresponding Atlan asset.
Database
Atlan always creates a Database asset called Hive to maintain the three-level hierarchy.
Schemas
Atlan maps schemas from Hive to its Schema asset type.
| Source property | Atlan property |
|---|---|
| DATABASE_NAME | name |
| COMMENT | description |
Tables
Atlan maps tables from Hive to its Table asset type.
| Source property | Atlan property |
|---|---|
| tableName | name |
| COMMENT | description |
| column_count | columnCount |
| numRows | rowCount |
| totalSize | sizeBytes |
| numPartitions | isPartitioned |
| numPartitions | partitionCount |
| location | externalLocation |
| inputFormat | externalLocationFormat |
Views
Atlan maps views from Hive to its View asset type.
| Source property | Atlan property |
|---|---|
| tableName | name |
| COMMENT | description |
| column_count | columnCount |
| viewExpandedText | definition |
Materialized views
Atlan maps materialized views in Hive to its Materialised View asset type.
| Source property | Atlan property |
|---|---|
| tableName | name |
| COMMENT | description |
| column_count | columnCount |
| viewExpandedText | definition |
| staleSinceDate | materializationTime |
| numRows | rowCount |
| totalSize | sizeBytes |
Columns
Atlan maps columns from Hive to its Column asset type.
| Source property | Atlan property |
|---|---|
| COL_NAME | name |
| DATA_TYPE | dataType |
| COMMENT | description |
| Detailed Table Information | order |
| Constraints (Primary Key) | isPrimary |
| Constraints (Foreign Key) | isForeign |