Skip to main content

What is gold layer

The Gold layer is the curated, analytics-ready tier of the Atlan Lakehouse. It transforms raw catalog metadata into standardized, queryable views for reporting, dashboards, and AI consumption.

Architecture​

The Lakehouse uses a medallion architecture with two primary layers:

  • Bronze layer: Raw metadata stored directly in your Atlan catalog. This includes unprocessed information about assets, lineage, tags, and governance as it's collected from your data sources.
  • Gold layer: Curated views that transform Bronze layer metadata into analytics-ready formats. This layer provides standardized schemas that are consistent across supported platforms.

The Bronze layer is the source of truth. The Gold layer is the consumption layer.

How the Gold layer works​

The Gold layer runs in your compute environment on top of the Lakehouse catalog. When you deploy the Gold layer, SQL views and tables are created in your platform and refreshed on a schedule that you control. You query these views using standard SQL just like any other database.

The Gold layer doesn't duplicate metadata. It provides a curated lens that makes metadata easier to query and analyze.

What is included​

The Gold layer provides standardized views across several metadata domains:

  • Asset registry: ASSETS and RELATIONAL_ASSET_DETAILS provide unified and detailed asset inventories.
  • Governance and classification: TAGS and CUSTOM_METADATA capture classifications and business-specific metadata.
  • Lineage and impact analysis: LINEAGE and LINEAGE_PROCESSES provide multi-hop lineage and transformation context.
  • Data quality: DATA_QUALITY_DETAILS captures data quality rules from Atlan and third-party tools.
  • Business glossary: GLOSSARY_DETAILS maps glossaries, categories, and terms to assets.
  • Pipeline and orchestration: PIPELINE_DETAILS consolidates pipeline metadata across orchestration tools.
  • Documentation: README provides searchable, cleaned documentation content.
  • Data mesh: DATA_MESH_DETAILS captures data domains and data products.

Why use the Gold layer​

The Gold layer makes metadata queryable at scale. It provides consistent schemas across platforms, supports analytics-ready queries, and enables automation and AI-driven workflows. Use it to build dashboards, generate compliance reports, and power metadata-driven applications.

Deployment model​

The Gold layer deploys in your compute environment. Your metadata stays within your security boundary, you control refresh schedules and costs, and you manage access using your platform permissions.

Platform support​

The Gold layer provides identical schemas across supported platforms:

  • Snowflake: Uses dynamic tables for automatic refresh.
  • Databricks: Uses Delta Live Tables for incremental updates.
  • BigQuery: Uses scheduled queries and materialized views.
  • DuckDB: Supports local development and testing.
  • Trino: Supports federated query across data sources.

Entity-relationship diagram​

An entity-relationship diagram for the Gold layer is shown below.

Lakehouse Gold layer ER diagram

How it fits with Atlan​

Use the Atlan UI for interactive exploration and collaborative governance. Use the Gold layer when you need programmatic access, custom analytics, or bulk operations on metadata.

See also​