Put Metadata Lakehouse to work

||Connect docs via MCP

The Metadata Lakehouse (MDLH) stores all of your tenant's metadata as managed tables that you query with plain SQL from your own compute engine. Atlan brings the storage and keeps it fresh; you bring the compute. This guide helps you decide when MDLH is the right surface, match your job to a proven pattern, and—for AI use cases—feed metadata to an agent in the way that stays accurate.

When to use what

MDLH is the right surface when you need to read metadata at scale—for analytics, reporting, extraction, or as context for AI. Match your job to a pattern rather than starting from a blank SQL prompt:

Your job	Pattern	Notes
Track enrichment / governance maturity	Enrichment / completeness reporting	The canonical AI-readiness scorecard; the headline metric is a domain-level enrichment score
Find what breaks or what's affected	Lineage RCA / impact analysis	Match the query to the access pattern—ad-hoc deep-dive, dashboard, or standard-depth
Bulk-export a large estate	Metadata export	The direct answer to native-export or API timeouts at volume
Identify cost-optimization opportunities	Usage / cost analysis	Often the most compelling leadership-facing entry point
Surface governance KPIs to leadership	Compliance / audit reporting	Powers any BI tool (Power BI, Tableau, Sigma)
Feed context to an AI agent	Talk-to-Data (a build pattern, not a toggle)	Build a curated semantic layer first—see below

It's a managed service—you manage nothing operationally. Atlan provisions the lakehouse, keeps it fresh, and maintains the underlying tables. The data lives in Atlan-managed cloud object storage matching your tenant's cloud; it's not copied onto your warehouse and never leaves the Atlan environment. To your users it simply appears as another database to query. One boundary to state clearly: the views you create on your own compute engine from the connection scripts are yours to maintain—when view definitions change, your team re-runs the definition scripts. Managed refers to the underlying tables, not to customer-side SQL infrastructure built on top.

Scope any replication or exposure deliberately. If you replicate lakehouse data into your own platform, default to a full-replace load while volumes are small and move to incremental only once full-replace runtime or cost becomes prohibitive. Expose only the curated gold-layer namespace to consumers—not raw tables—and add curated gold tables for net-new use cases instead of widening exposure to raw tables.

Lead with the "why" before any query. The alternative—exporting all metadata to a flat file and loading it into your platform, or polling the API—is slow (hours to days for millions of assets) and strains production. MDLH decouples consumption from the application layer. Teams that don't grasp the "why" treat MDLH as a curiosity and defer the connection work.

MDLH is a prerequisite for AI, not an afterthought. Context Studio depends on it—confirm the current prerequisite list in the product documentation—and it makes the MCP more effective. When AI or security approval is pending, treat MDLH as the productive parallel track—it delivers standalone value (metadata analytics, usage reporting) on its own.

Recommended sequence

1. Enable the lakehouse. Enablement is self-serve through the in-product Marketplace workflow; confirm the current steps—and where to copy the connection details—in the product documentation.
2. Connect your compute engine. The Atlan side is ready immediately—the tables are already running in your tenant. The work is on your side: connect a supported compute engine (common choices include Snowflake, Databricks, and BigQuery; a no-cost fallback exists): confirm the current supported-engine list in the Atlan product documentation. Some engines have setup specifics (recursive-query support differs; some need a flag enabled on the compute side; some do namespace-level discovery you must extend per new schema). Get the latest connection script from Atlan directly—documentation can lag the script cycle.
3. Match your job to a pattern from the preceding table before writing SQL. Start from the pre-built query library, not the raw schema—walk one concrete query end-to-end (completeness tracking is a strong first choice because it maps to a visible governance goal), and treat the library as a starting point to tweak.
4. Match the lineage access pattern to the query. On a large tenant, a naive multi-hop lineage query evaluates the whole graph before filtering and looks slow. Use the per-asset function for ad-hoc lookups, a materialized table for dashboards, and the standard traversal for bounded-depth needs. Route the exact query model and cross-engine syntax to the gold-layer query reference.
5. For AI use cases, build the curated semantic layer first (see below), then connect it to your model.

Feed metadata to AI right way

For any "feed Atlan context to AI" effort—Cortex, Databricks Genie, Gemini, Claude via MCP, or a custom model—the durable lesson is: never point an agent at raw metadata tables.

Raw metadata is complex: hundreds of table types, hundreds of columns per table, millions of assets across warehouses, BI, pipelines, quality, glossary, and object storage. A generic agent can't reliably infer joins, pick the right tables, traverse lineage, or resolve internal identifiers to human-readable names. Accuracy demands a curated, well-keyed semantic layer that guides the agent—a clean, domain-aligned set of views that flattens tags and custom metadata, resolves glossary, and expands lineage, plus a semantic model with explicit join rules and a set of verified example queries the model can learn from.

The reference sequence is: curated semantic layer → semantic model → MCP server → agent. Talk-to-Data is a build pattern a solutions effort stands up, not an out-of-the-box toggle—treat it as a proof-of-concept blueprint rather than a shipped feature.

Atlan's context tooling is complementary to your warehouse's native AI, not a replacement. If your agents always live inside one warehouse's assistant (for example, Cortex or Genie), use that assistant. The case for building context in Atlan is (1) cross-source context—one semantic layer spanning warehouses, BI, and pipelines; (2) portability—context expressed as plain artifacts (YAML, DDL, skill documents) deploys to any agent runtime without lock-in; and (3) a place to test and iterate on context before wiring it to a production agent. Being clear about this boundary keeps the decision honest.

Sending metadata back to source systems: rank the options. When a team wants Atlan-authored context written back to the source (or a two-way flow), present the options in preference order: (1) MCP plus an AI coding agent—cleanest, but needs your security approval for agent tooling; (2) a scheduled job through the Atlan API—query the source's information schema, diff, and upload on a cadence; a practical interim while agent approval is pending; (3) file-based import utilities—a last resort: they match on exact qualified names and give no diagnostic output on a near-miss. Confirm the current scope of native reverse-sync in the product documentation before promising field-level parity—historically it has covered a narrow slice (primarily tags), and push-back to source is a separately deployed component.

Sequence MDLH ahead of the AI conversation. Get MDLH connected before you conclude the AI discussion, and set up the Context Studio infrastructure in the tenant as soon as MDLH is ready—so you can act on approval day one. When AI features are blocked on security review, keep MDLH moving in parallel for its standalone value.

Common pitfalls

• Jumping to a query demo before establishing the architectural "why": your team defers the connection work.
• Sending the schema docs and leaving the team to explore the raw tables alone: overwhelm, and slow time-to-first-query. Walk one concrete query live.
• Pointing an AI agent at raw metadata tables: unreliable joins and identifier-resolution failures. Build the curated semantic layer first.
• Letting a naive lineage query on a very large tenant run to completion and concluding MDLH is slow: match the access pattern to the query instead.
• Not checking which schemas or namespaces your compute connection covers: new schemas can go silently unmapped.
• Promising full reverse-sync parity or native in-platform custom KPI dashboards: confirm what actually ships in the product documentation before you commit, and share any gaps with your Atlan account team as product feedback.
• Judging MDLH on storage-engine internals: those are implementation details, not reasons to adopt.

Troubleshooting

Symptom	Likely cause	Next move
Lakehouse enablement or the compute connection isn't working	A tenant-enablement or connection-detail issue, not a query-config error	Confirm the lakehouse is enabled and re-copy the connection details from the Marketplace view; if enablement itself fails or stalls, open a support ticket with the error (volumes, mechanism, error text).
Lineage queries are slow	SQL-engine behavior on a large tenant, not a bug	Match the access pattern: per-asset function for ad-hoc, materialized table for dashboards, standard traversal for bounded depth. See the gold-layer query reference.
Access denied on managed views but not on tables	Permissions on the underlying Atlan-owned tables the views depend on	Test the suspected underlying tables individually to isolate scope, then escalate the ticket with your findings.
A bulk export times out	Native/API export can't handle the volume	This is MDLH's direct unblock—extract from SQL at scale instead.
A custom job fails at very high operation counts	An engineering escalation, not config advice	Capture operation counts, mechanism, and the error in writing and raise it with Atlan support—this is an engineering issue, not a config change.
A source object was dropped and seems to have "disappeared"	Dropping a source object soft-deletes (archives) it	It still counts and is recoverable; raise audit-retention before any purge.
A team wants 40–50 KPIs shown inside the product	Atlan's standard reporting module may not cover a fully customizable in-platform KPI layout	Confirm the current in-platform reporting capability in the product documentation, then power a BI tool from MDLH. First confirm you can actually query MDLH—a permissions check is the common silent blocker. Share the in-platform requirement with your Atlan account team as product feedback.
Table names in your lakehouse don't match the documented schema (for example, mixed-case names)	The tenant was onboarded before a naming-convention standardization; the mismatch compounds as new capabilities depend on the documented schema	Plan a re-onboarding to the current convention before the next major dependent use case goes live—once classification or AI grounding runs on top, multi-day downtime becomes operationally unacceptable. Deferring doesn't avoid the rebuild; even later platform changes still require it. Gate the timing on urgency, and shrink the rebuild window first by hard-deleting content you no longer need.

For anything involving the gold-layer schema, table and column names, lineage query model, or SQL, route to the gold-layer query reference—that content is owned there, not here. For provisioning status, connector/compute coverage, refresh cadence, and GA-versus-preview facts, route to the Atlan product documentation.

Related

• The gold-layer query reference: the schema, tables, lineage query model, and SQL (owned there).
• API, SDK & automation: the API-polling alternative MDLH replaces for bulk reads, and the MCP that MDLH makes more effective.
• Data products & domains: a downstream consumer of the curated metadata you expose.
• Metadata enrichment: AI-generated context that surfaces in the lakehouse for review and export.