Skip to main content
Community Hub

Extract Databricks Genie Agent lineage

TL;DR

Build lineage from Databricks Genie Agents to the Unity Catalog tables and columns they read.

Your AI can read this via Docs MCPInstall MCP →Connect
Private Preview

Databricks Genie Agent support is in private preview and is enabled per tenant. The Import Genie Agents and Enable Genie Agent Lineage options stay hidden in the workflow setup until the feature is enabled for your tenant. To request access, raise a support request.

Once you have crawled Databricks Genie Agents, Atlan can build lineage connecting each Genie Agent to the Unity Catalog tables and columns it reads.

Genie Agent lineage runs in the Databricks lineage and usage workflow—the same workflow that builds table and column lineage—not in the crawler.

Prerequisites​

Before extracting Genie Agent lineage, make sure you have:

  • Feature access enabled for your tenant: The Import Genie Agents and Enable Genie Agent Lineage options stay hidden in the workflow setup until the feature is enabled. Raise a support request to request access.
  • Crawled Databricks Genie Agents at least once—lineage for a Genie space Atlan hasn't cataloged produces no edges
  • Set up lineage and usage extraction for the same connection
  • SELECT on system.access.table_lineage and system.access.column_lineage

Extract lineage​

To extract Genie Agent lineage:

  1. Follow the standard Extract lineage and usage from Databricks steps.
  2. Under lineage extraction, turn on Enable Genie Agent Lineage.
  3. Run the workflow.
  4. After the workflow completes, open any Genie Agent asset in Atlan and select the Lineage tab to view its upstream tables and columns.

What lineage Atlan builds​

Atlan reads Genie Agent lineage from the Unity Catalog lineage system tables and creates these relationships:

Upstream assetDownstream assetSource
TableGenie Agentsystem.access.table_lineage
ViewGenie Agentsystem.access.table_lineage
ColumnGenie Agentsystem.access.column_lineage

Column lineage terminates at the Genie Agent itself. Databricks reports which source columns a Genie space reads, but not which part of the space consumes them, so Atlan doesn't invent a target column.

Atlan doesn't parse a Genie space's example SQL. All lineage comes from what Unity Catalog recorded.

Genie Agent lineage is best effort

Unity Catalog records lineage only for queries it observed. A Genie space nobody has queried, or one whose queries Databricks hasn't recorded yet, produces no lineage edges. An absent edge isn't a crawl failure.

See also​