Skip to main content

What is Context Engineering Studio? Private Preview

Context Engineering Studio (CES) is Atlan's environment for building, testing, and deploying the semantic context that AI agents need to answer business questions accurately. You start from your existing catalog (Snowflake schemas, dbt models, Sigma workbooks, Looker Explores, query history, and your Atlan glossary) and end with a certified, versioned semantic layer that grounds AI agents in your business definitions.

  • From your catalog, not from scratch. CES reads your existing metadata to generate an initial semantic model in hours. What used to take months of manual YAML authoring is ready the same day.
  • A loop that exposes what to fix. Every simulation surfaces specific gaps (missing synonyms, ambiguous metrics, under-specified joins) and proposes the change. You learn where the context needs more work before real users see it.
  • One definition, every AI surface. A single certified context repository deploys to Snowflake Cortex Analyst, Databricks Genie, or any MCP-compatible agent. Update a metric definition once and every agent that uses it reflects the change immediately.

How it works​

CES structures context engineering as a three-step loop (Build, Simulate, Deploy) that runs continuously as your data and business evolve. Observe is currently available for Snowflake Cortex deployments as the feedback channel that feeds the next loop.

Build​

Connect your Snowflake or Databricks account and describe your domain in plain English, for example: "Sales pipeline and revenue metrics for the go-to-market team." CES searches your Atlan catalog and surfaces the tables and columns most relevant to that domain, ranked by usage, lineage coverage, and glossary matches.

Once you confirm the asset and column selection, Context Agents Studio runs specialized agents in parallel to generate the initial semantic model:

  • Descriptions for every asset and column
  • Synonyms for business terms users might phrase differently
  • Metrics inferred from connected BI tools (Sigma, Looker, Tableau) and query history
  • Filters inferred from operational query patterns
  • Relationships recommended from lineage and query history
  • Custom instructions that carry domain nuance
  • Primary keys detected from schema and usage signals

Descriptions and synonyms are applied automatically. They're additive changes that don't affect the model's structure or behavior. Metrics, filters, relationships, custom instructions, and primary key suggestions queue for your review before entering the model, because these affect how the agent answers questions.

Use Chat & build to test natural-language questions on the live model, request structural changes in plain English, or edit the underlying YAML directly.

Simulate​

Simulate shows you where the model needs more context. You assemble a question set, run it on the semantic model, and read the results to see which questions land, which miss, and why each miss happened.

Each simulation produces diagnostic signals you can act on:

  • Paraphrases of the same question produce different answers: a synonym is missing.
  • The agent chose an adjacent table instead of the canonical one: the preferred asset isn't flagged strongly enough.
  • A metric returns different numbers under different filters: two competing definitions are in the model.
  • A question can't be answered at all: an asset or relationship is out of scope.

Every miss is paired with a proposed change. Apply the fix, re-run the suite, and confirm whether the specific gap closed without regressing anything that was working. Simulation is a learning loop, not a grading function.

Deploy​

When the model is answering the questions the business actually asks, certify the context repository. Certification locks the model to a versioned snapshot with a contributor log and timestamp. Subsequent changes require a new review cycle before they take effect.

CES then generates the deployment artifact for your target engine and pushes it directly:

  • Snowflake Cortex Analyst: a Semantic View in your target database and schema.
  • Databricks Genie: a Metric View in your catalog and schema, plus a Genie Space pre-populated with your business context.

After deployment, business users can immediately ask natural-language questions through the AI interface of their choice.

Observe​

Observe is currently available for Snowflake Cortex deployments.

The Observe tab surfaces production query traces from Cortex Analyst: every question a user asked, the SQL the engine generated, the result, and any feedback the user left.

For each trace you can:

  • Promote a successful interaction to the question set as a verified question-answer pair.
  • Suggest fixes for a failing question and get AI-proposed YAML changes.
  • Apply and retest approved fixes without leaving CES.
  • Extract terms from production queries back into your Atlan glossary.

Production failures become question set entries, which drive the next certification cycle. The model improves continuously from real usage, not from manual audits.

On Databricks, close the feedback loop by reviewing Genie's conversation history directly in the workspace and adding representative failures back into your question set manually.

Key concepts​

Context repository​

A context repository is a bounded, versioned package of semantic context for a specific domain. It contains the table and column definitions, business logic, verified question-answer pairs, and deployment artifacts an AI agent needs to answer questions about that domain.

Repositories move through a three-stage lifecycle:

StageMeaning
DraftBeing built, refined, and tested
ActiveCertified and deployed to at least one target
ArchivedRetired and no longer served to agents

Changes to an active repository require a new draft cycle and re-certification before they take effect. All changes are logged with the contributor and timestamp.

Sessions and repositories​

A session is your in-progress working state inside CES: the assets you've selected, the questions you've added, the simulation results from your last run, and the current YAML in the editor. Sessions persist automatically so you can close the browser and pick up where you left off.

A repository is a named, savable snapshot of a session. Repositories survive across sessions and can be reopened to continue building. Repositories carry their engine context (connector type, deployed view name on Snowflake, Genie Space ID on Databricks), so loading a deployed repository into a new session immediately routes Chat & build and Simulate to the right live artifact without a re-deploy.

Semantic model​

The semantic model is the YAML definition of business logic for a domain. It specifies which tables are in scope, how they join, how metrics are calculated, which filters apply by default, and which synonyms map to each business term.

CES stores the model as editable YAML and compiles it to the format each engine requires at deploy time. You can edit it through Chat & build (in plain English), through the YAML editor directly, or let Context Agents Studio propose changes that you review and approve.

See the YAML schema reference for the complete field reference for both Snowflake and Databricks.

Question set​

A question set is a collection of natural-language business questions, each paired with verified SQL written by a domain expert. You run the question set through a simulation to measure how the semantic model handles real business queries.

A simulation surfaces signals across several dimensions:

  • Coverage: which questions can the model answer? Which fall outside its current scope?
  • Grounding: did the agent use the right assets, join paths, and metric definitions, or did it arrive at a correct-looking answer through the wrong route?
  • Consistency: does the same question, asked in different phrasings, produce the same answer?
  • Failure shape: when something's wrong, is it loud and diagnosable (missing term, unresolved join) or subtle?

Each dimension maps to a different type of context gap. A coverage miss means an asset is out of scope; a grounding miss means a relationship is under-specified; a consistency miss usually points to a missing synonym. Simulation connects those gaps to specific changes in the model.

Start with 10 to 20 questions that represent the most common queries for the domain, then grow the set with interactions promoted from the Observe tab or added manually from Genie conversation history.

Context Agents Studio​

Context Agents Studio is CES's enrichment assistant. When you add assets to a context repository, it proposes descriptions, glossary term mappings, join recommendations from lineage and query history, metric definitions inferred from your BI layer, and synonym suggestions. Enrichment is prioritized by asset usage frequency so high-traffic tables are covered first.

Every proposal requires your review before it enters the model.

Observe​

Observe surfaces production query traces from your deployed Snowflake Cortex Analyst. Each trace shows the user's question, the SQL Cortex Analyst generated, the result, and any user feedback. Traces you promote become part of the question set, closing the feedback loop between production failures and the next certification cycle.

Observe is currently available for Snowflake Cortex deployments.

Deployment targets​

TargetWhat gets deployed
Snowflake Cortex AnalystSemantic View in your target database and schema
Databricks GenieMetric View in your catalog and schema, plus a pre-populated Genie Space

See also​