What does Atlan crawl from Microsoft Azure Cosmos DB?
Once you have crawled Microsoft Azure Cosmos DB, you can use connector-specific filters for quick asset discovery. The following filters are currently supported for Microsoft Azure Cosmos DB assets:
- Accounts - you can use the following filters:
- Account Type - filter accounts by account type
- Subscription ID - filter accounts by subscription ID
- Account Resource Group - filter accounts by resource group
- Collections - Type filter, filtering options include Time Series, Capped, and Clustered collection types
Atlan crawls and maps the following assets and properties from Microsoft Azure Cosmos DB. Atlan currently does not support lineage for Microsoft Azure Cosmos DB assets.
Accounts
Atlan maps accounts from Microsoft Azure Cosmos DB to its Account asset type.
vCore-based deployment
| Source property | Atlan property |
|---|---|
name | name |
type | cosmosMongoDBAccountType |
| dynamically generated | cosmosMongoDBDatabaseCount |
RU-based deployment
| Source property | Atlan property |
|---|---|
name | name |
createdAt | sourceCreatedAt |
instanceId | cosmosMongoDBAccountInstanceId |
| dynamically generated | cosmosMongoDBDatabaseCount |
type | cosmosMongoDBAccountType |
subscription | cosmosMongoDBAccountSubscriptionId |
resource group | cosmosMongoDBAccountResourceGroup |
document endpoint | cosmosMongoDBAccountDocumentEndpoint |
mongo endpoint | cosmosMongoDBAccountMongoEndpoint |
publicNetworkAccess | cosmosMongoDBAccountPublicNetworkAccess |
enableAutomaticFailover | cosmosMongoDBAccountEnableAutomaticFailover |
enableMultipleWriteLocations | cosmosMongoDBAccountEnableMultipleWriteLocations |
enablePartitionKeyMonitor | cosmosMongoDBAccountEnablePartitionKeyMonitor |
isVirtualNetworkFilterEnabled | cosmosMongoDBAccountIsVirtualNetworkFilterEnabled |
locations | cosmosMongoDBAccountLocations |
readLocations | cosmosMongoDBAccountReadLocations |
writeLocations | cosmosMongoDBAccountWriteLocations |
defaultConsistencyLevel | cosmosMongoDBAccountConsistencyPolicy |
Databases
Atlan maps databases from Microsoft Azure Cosmos DB to its Database asset type.
| Source property | Atlan property |
|---|---|
TABLE_CAT | name |
TABLE_COUNT | mongoDBDatabaseCollectionCount |
Collections
Atlan maps collections from Microsoft Azure Cosmos DB to its Collection asset type.
| Source property | Atlan property |
|---|---|
TABLE_NAME | name |
TABLE_CAT | databaseName |
COLUMN_COUNT | columnCount |
collStats.storageSize | sizeBytes |
collStats.count | rowCount |
type | mongoDBCollectionSubtype |
schema | mongoDBCollectionSchemaDefinition |
collStats.capped | mongoDBCollectionIsCapped |
collStats.maxSize | mongoDBCollectionMaxSize |
collStats.max | mongoDBCollectionMaximumDocumentCount |
options.timeseries.timeField | mongoDBCollectionTimeField |
options.timeseries.granularity | mongoDBCollectionTimeGranularity |
options.expireAfterSeconds | mongoDBCollectionExpireAfterSeconds |
collStats.numOrphanDocs | mongoDBCollectionNumOrphanDocs |
collStats.nindexes | mongoDBCollectionNumIndexes |
collStats.totalIndexSize | mongoDBCollectionTotalIndexSize |
collStats.avgObjSize | mongoDBCollectionAverageObjectSize |
Columns
To extract columns from Microsoft Azure Cosmos DB collections, Atlan analyzes document schemas, which must be enabled and configured to crawl columns.
Atlan consolidates multiple document schemas into a unified view, capturing all possible fields with their nesting levels and data types. The system then performs a depth-first traversal, converting each unique field path into column entries while tracking parent-child relationships and hierarchy through qualified names. The total column count is calculated by summing up all unique paths, including nested structures, with arrays processed to identify additional column patterns.
Atlan maps the following metadata from Microsoft Azure Cosmos DB to its Column asset type.
- Atlan supports nested columns up to level 35 for parent columns.
- Column-level lineage is currently not supported.
- Tag propagation is supported from:
- collections to columns
- parent to nested columns
| Atlan property | Where in Atlan |
|---|---|
name | asset preview and profile, overview sidebar |
order | asset preview and profile, overview sidebar |
dataType | asset preview and profile, overview sidebar |
rawdataTypeDefinition(raw schema of nested columns in a given parent column) | asset preview and profile, overview sidebar |
path(complete hierarchy from parent column to child column) | asset preview and profile, overview sidebar |