What does Atlan crawl from Microsoft Azure Data Factory?

Atlan crawls and maps the following assets and properties from Microsoft Azure Data Factory.

For any currently unsupported linked services, datasets, and activities from Microsoft Azure Data Factory not listed below, Atlan will map them to the relevant asset type and only display asset name, type, and description.

Linked services

Atlan maps linked services from Microsoft Azure Data Factory to its AdfLinkedservice asset type.

Microsoft Azure Cosmos DB for MongoDB

Source property	Atlan property
`name`	`name`
`type`	`adfLinkedserviceType`
`description`	`description`
`annotations`	`adfLinkedserviceAnnotations`
`typeProperties.database`	`adfLinkedserviceDatabaseName`
`typeProperties.isServerVersionAbove32`	`adfLinkedserviceVersionAbove`
`version`	`adfLinkedserviceVersion`

Azure Data Lake Storage (ADLS)

Source property	Atlan property
`name`	`name`
`type`	`adfLinkedserviceType`
`description`	`description`
`typeProperties.azureCloudType`	`adfLinkedserviceAzureCloudType`
`typeProperties.servicePrincipalCredentialType`	`adfLinkedserviceCredentialType`
`typeProperties.tenant`	`adfLinkedserviceTenant`
`typeProperties.url`	`adfLinkedserviceDomainEndpoint`

Azure Databricks

Source property	Atlan property
`name`	`name`
`type`	`adfLinkedserviceType`
`description`	`description`
`annotations`	`adfLinkedserviceAnnotations`
`typeProperties.clusterId`	`adfLinkedserviceClusterId`
`typeProperties.domain`	`adfLinkedserviceDomainEndpoint`
`typeProperties.workspaceResourceId`	`adfLinkedserviceResourceId`

Snowflake

Source property	Atlan property
`name`	`name`
`type`	`adfLinkedserviceType`
`description`	`description`
`annotations`	`adfLinkedserviceAnnotations`
`typeProperties.accountIdentifier`	`adfLinkedserviceDomainEndpoint`
`typeProperties.authenticationType`	`adfLinkedserviceCredentialType`
`typeProperties.database`	`adfLinkedserviceDatabaseName`
`typeProperties.user`	`adfLinkedserviceUserName`
`typeProperties.warehouse`	`adfLinkedserviceWarehouseName`

Microsoft Azure SQL Database

Source property	Atlan property
`name`	`name`
`type`	`adfLinkedserviceType`
`description`	`description`
`annotations`	`adfLinkedserviceAnnotations`
`typeProperties.authenticationType`	`adfLinkedserviceCredentialType`
`typeProperties.database`	`adfLinkedserviceDatabaseName`
`typeProperties.server`	`adfLinkedserviceDomainEndpoint`

Datasets

Atlan maps databases from Microsoft Azure Cosmos DB to its AdfDataset asset type.

Microsoft Azure Cosmos DB for MongoDB

Source property	Atlan property
`name`	`name`
`type`	`adfDatasetType`
`description`	`description`
`annotations`	`adfDatasetAnnotations`
`folder`	`adfDatasetFolderPath`
`linkedServiceName`	`adfDatasetLinkedService`
`typeProperties.collection`	`adfDatasetCollectionName`

Azure Data Lake Storage (ADLS)

Source property	Atlan property
`name`	`name`
`type`	`adfDatasetType`
`description`	`description`
`annotations`	`adfDatasetAnnotations`
`folder`	`adfDatasetFolderPath`
`linkedServiceName`	`adfDatasetLinkedService`
`typeProperties.fileName`	`adfDatasetFileName`
`typeProperties.folderPath`	`adfDatasetFileFolderPath`
`typeProperties.format`	`adfDatasetStorageType`
`typeProperties.container`	`adfDatasetContainerName`

Azure Databricks

Source property	Atlan property
`name`	`name`
`type`	`adfDatasetType`
`description`	`description`
`annotations`	`adfDatasetAnnotations`
`folder`	`adfDatasetFolderPath`
`linkedServiceName`	`adfDatasetLinkedService`
`typeProperties.database`	`adfDatasetDatabaseName`
`typeProperties.table`	`adfDatasetTableName`

Snowflake

Source property	Atlan property
`name`	`name`
`type`	`adfDatasetType`
`description`	`description`
`annotations`	`adfDatasetAnnotations`
`folder`	`adfDatasetFolderPath`
`linkedServiceName`	`adfDatasetLinkedService`
`typeProperties.schema`	`adfDatasetSchemaName`
`typeProperties.table`	`adfDatasetTableName`

Microsoft Azure SQL Database

Source property	Atlan property
`name`	`name`
`type`	`adfDatasetType`
`description`	`description`
`annotations`	`adfDatasetAnnotations`
`folder`	`adfDatasetFolderPath`
`linkedServiceName`	`adfDatasetLinkedService`
`typeProperties.schema`	`adfDatasetSchemaName`
`typeProperties.table`	`adfDatasetTableName`

Data flows

Atlan maps data flows from Microsoft Azure Data Factory to its AdfDataflow asset type.

Source property	Atlan property
`name`	`name`
`description`	`description`
`folder`	`adfDataflowFolderPath`
`typeProperties.sources`	`adfDataflowSources`
`typeProperties.sinks`	`adfDataflowSinks`
`typeProperties.scriptLines`	`adfDataflowScript`

Activities

Atlan maps activities from Microsoft Azure Data Factory to its AdfActivity asset type.

Copy activity

Source property	Atlan property
`name`	`name`
`type`	`adfActivityType`
`description`	`description`
`dependsOn`	`adfActivityPrecedingDependency`
`policyTimeout`	`adfActivityPolicyTimeout`
`policyRetryInterval`	`adfActivityPolictRetryInterval`
`state`	`adfActivityState`
`activityRuns`	`adfActivityRuns`
`typeProperties.sink`	`adfActivitySinks`
`typeProperties.source`	`adfActivitySources`
`typeProperties.sinkType`	`adfActivitySinkType`
`typeProperties.sourceType`	`adfActivitySourceType`

Databricks notebooks

Source property	Atlan property
`name`	`name`
`type`	`adfActivityType`
`description`	`description`
`dependsOn`	`adfActivityPrecedingDependency`
`policyTimeout`	`adfActivityPolicyTimeout`
`policyRetryInterval`	`adfActivityPolictRetryInterval`
`state`	`adfActivityState`
`activityRuns`	`adfActivityRuns`
`typeProperties.source`	`adfActivitySources`
`typeProperties.sourceType`	`adfActivitySources`
`typeProperties.notebookPath`	`adfActivityNotebookPath`

DatabricksSparkJar activity

Source property	Atlan property
`name`	`name`
`type`	`adfActivityType`
`description`	`description`
`dependsOn`	`adfActivityPrecedingDependency`
`policyTimeout`	`adfActivityPolicyTimeout`
`policyRetryInterval`	`adfActivityPolictRetryInterval`
`state`	`adfActivityState`
`activityRuns`	`adfActivityRuns`
`typeProperties.source`	`adfActivitySources`
`typeProperties.sourceType`	`adfActivitySources`
`typeProperties.mainClassName`	`adfActivityMainClassName`

DatabricksSparkPython activity

Source property	Atlan property
`name`	`name`
`type`	`adfActivityType`
`description`	`description`
`dependsOn`	`adfActivityPrecedingDependency`
`policyTimeout`	`adfActivityPolicyTimeout`
`policyRetryInterval`	`adfActivityPolictRetryInterval`
`state`	`adfActivityState`
`activityRuns`	`adfActivityRuns`
`typeProperties.source`	`adfActivitySources`
`typeProperties.sourceType`	`adfActivitySources`
`typeProperties.pythonFile`	`adfActivityPythonFilePath`

Lookup activity

Source property	Atlan property
`name`	`name`
`type`	`adfActivityType`
`description`	`description`
`dependsOn`	`adfActivityPrecedingDependency`
`policyTimeout`	`adfActivityPolicyTimeout`
`policyRetryInterval`	`adfActivityPolictRetryInterval`
`state`	`adfActivityState`
`activityRuns`	`adfActivityRuns`
`typeProperties.firstRowOnly`	`adfActivityFirstRowOnly`
`typeProperties.source`	`adfActivitySources`

ForEach activity

Source property	Atlan property
`name`	`name`
`type`	`adfActivityType`
`description`	`description`
`dependsOn`	`adfActivityPrecedingDependency`
`policyTimeout`	`adfActivityPolicyTimeout`
`policyRetryInterval`	`adfActivityPolictRetryInterval`
`state`	`adfActivityState`
`activityRuns`	`adfActivityRuns`
`typeProperties.activities`	`adfActivitySubActivities`
`typeProperties.batchCount`	`adfActivityBatchCount`
`typeProperties.isSequential`	`adfActivityIsSequential`

Pipelines

Atlan maps pipelines from Microsoft Azure Data Factory to its AdfPipeline asset type.

Source property	Atlan property
`name`	`name`
`type`	`adfActivityType`
`description`	`description`
`activityCount`	`adfPipelineActivityCount`
`pipelineRun`	`adfPipelineRuns`
`typeProperties.folder`	`adfPipelineFolderPath`

Linked services​

Microsoft Azure Cosmos DB for MongoDB​

Azure Data Lake Storage (ADLS)​

Azure Databricks​

Snowflake​

Microsoft Azure SQL Database​

Datasets​

Microsoft Azure Cosmos DB for MongoDB​

Azure Data Lake Storage (ADLS)​

Azure Databricks​

Snowflake​

Microsoft Azure SQL Database​

Data flows​

Activities​

Copy activity​

Databricks notebooks​

DatabricksSparkJar activity​

DatabricksSparkPython activity​

Lookup activity​

ForEach activity​

Pipelines​

Linked services

Microsoft Azure Cosmos DB for MongoDB

Azure Data Lake Storage (ADLS)

Azure Databricks

Snowflake

Microsoft Azure SQL Database

Datasets

Microsoft Azure Cosmos DB for MongoDB

Azure Data Lake Storage (ADLS)

Azure Databricks

Snowflake

Microsoft Azure SQL Database

Data flows

Activities

Copy activity

Databricks notebooks

DatabricksSparkJar activity

DatabricksSparkPython activity

Lookup activity

ForEach activity

Pipelines