Skip to main content
Community Hub

How Atlan builds lineage for Databricks Genie Agents

TL;DR

Frequently asked questions about how Atlan catalogs Databricks Genie spaces and constructs their lineage, including lineage sources, relationships created, and known limits.

Your AI can read this via Docs MCPInstall MCP →Connect

This FAQ covers how Atlan catalogs Databricks Genie spaces, how it constructs their lineage, and what the known limits are.

Is Databricks Genie Agent support generally available?​

No. Genie Agent support is in private preview and is enabled per tenant. The Import Genie Agents, Probe Genie Serialized Detail, and Enable Genie Agent Lineage options stay hidden in the workflow setup until the feature is enabled for your tenant. To request access, raise a support request.

What does Atlan create for each Databricks Genie space?​

One Genie Agent asset for each Genie space, published under the same connection as your other Databricks assets. Its Atlan asset type is DatabricksGenieAgent.

How does Atlan build lineage for Genie Agents?​

Atlan reads Genie Agent lineage from the Unity Catalog lineage system tables, system.access.table_lineage and system.access.column_lineage. Databricks records which tables and columns each Genie space reads, and Atlan turns those records into lineage edges pointing at the cataloged Genie Agent asset.

What lineage relationships does Atlan create for Genie Agents?​

Upstream assetDownstream assetSource
TableGenie Agentsystem.access.table_lineage
ViewGenie Agentsystem.access.table_lineage
ColumnGenie Agentsystem.access.column_lineage

Why doesn't column lineage point at specific parts of Genie spaces?​

Databricks reports which source columns a Genie space reads, but not which question, instruction, or saved query consumes each one. Atlan doesn't invent a target it can't verify, so column lineage terminates at the Genie Agent asset itself.

Why don't I see lineage for Genie Agents Atlan has cataloged?​

Lineage and Genie metadata come from different places, so they can be out of step:

  • Lineage extraction hasn't run since the Genie space was crawled. Genie metadata comes from the crawler. Lineage comes from the lineage and usage workflow. Run the lineage workflow after crawling.
  • Nobody has queried the Genie space. Unity Catalog records lineage when a query runs. A Genie space with no query activity has no lineage rows to read.
  • Databricks hasn't recorded the lineage yet. Unity Catalog populates the lineage system tables asynchronously, so recent Genie activity might not appear in the next Atlan run.

Genie Agent lineage is best effort. An absent edge isn't a crawl failure.

Why are change tokens empty on my Genie Agents?​

Atlan reads the change token only when Probe Genie Serialized Detail is turned on, and that read requires CAN EDIT on every Genie space. When the option is off, or the service account doesn't hold CAN EDIT, Atlan catalogs the space and leaves the change token empty rather than failing the crawl.

Does Atlan read Genie space instructions or example SQL?​

No. Atlan catalogs a Genie space's identity and configuration references only. It doesn't read, store, or emit instructions, sample questions, example SQL, verified answers, or the serialized body of the space.

What happens to my Genie Agents if I turn off Import Genie Agents?​

The next run removes them from Atlan. The crawler stops extracting Genie spaces, so they're absent from the snapshot Atlan compares against the previous run, and Atlan archives them.

Turning the option back on re-crawls your Genie spaces, and they reappear with their metadata on the next run. Their lineage isn't restored by the crawl—run the lineage and usage workflow with Enable Genie Agent Lineage turned on to rebuild it.

How do Import Genie Agents and Import Genie Code Skills differ?​

Import Genie Agents catalogs Genie spaces as Genie Agent assets. Import Genie Code Skills catalogs SKILL.md files as Skill assets. They're separate options and neither one turns on the other.

See also​