Skip to main content

What does Atlan crawl from Apache Airflow/OpenLineage?

Once you have integrated Apache Airflow/OpenLineage, you can use connector-specific filters for quick asset discovery. The following filters are currently supported:

  • Status filter - last run status for an asset
  • Duration filter - last run duration for an asset

Atlan maps the following assets and properties from Apache Airflow/OpenLineage. Asset lineage support depends on the list of operators supported by OpenLineage.

DAGs

Atlan maps DAGs (directed acyclic graphs) from Apache Airflow/OpenLineage to its AirflowDAG asset type.

Source propertyAtlan property
schedule_intervalairflowDagSchedule
timetableairflowDagScheduleDelta
tagsairflowTags
descriptiondescription
ownersourceOwners
versionairflowRunVersion
openlineageAdapterVersionairflowRunOpenLineageVersion
runidairflowRunName
run_typeairflowRunType
eventTimeairflowRunStartTime
eventTimeairflowRunEndTime
eventTypeairflowRunOpenLineageState
queryairflowTaskSql
group_idairflowTaskGroupName
Did you know?

If a DAG has more than 10 valid owner email addresses (comma-separated), only the first 10 will be captured and published.

Tasks

Atlan maps tasks from Apache Airflow/OpenLineage to its AirflowTask asset type.

Source propertyAtlan property
retriesairflowTaskRetryNumber
poolairflowTaskPool
pool_slotsairflowTaskPoolSlots
queueairflowTaskQueue
priority_weightairflowTaskPriorityWeight
trigger_ruleairflowTaskTriggerRule
operator_classairflowTaskOperatorClass
dag_idairflowDagName
conn_id`airflowTaskConnectionId`
sqlairflowTaskSql
tagsairflowTags
versionairflowRunVersion
openlineageAdapterVersionairflowRunOpenLineageVersion
runidairflowRunName
run_typeairflowRunType
eventTimeairflowRunStartTime
eventTimeairflowRunEndTime
eventTypeairflowRunOpenLineageState