Run simulations Private Preview

This guide walks you through running simulations on your semantic model, reading the results, and applying fixes. Each simulation run surfaces where the model answers well and where it needs more context, including descriptions, joins, filters, or additional assets.

Prerequisites

Before you begin, make sure:

• You have a context repository with an initial semantic model. See Build your context repository.
• You've already created your question set. See Create your question set.
• If you're using Databricks, your context repository is deployed. Simulate uses the live Genie Space, not the draft. See Deploy to Databricks.

Getting started

1. In Context Studio, open your context repository and click the Simulate tab.

2. Click Pick a collection, then select a collection from the list. Collections are auto-generated based on your repository's assets, metrics, and domain scope. Each collection contains representative questions that test your semantic model's coverage and accuracy.

3. Once selected, the simulation begins. This might take a few minutes. When complete, you'll see:

   • Overall Score — Percentage of questions answered correctly and number passing
   • Test Results — Individual results for each question, including accuracy score, latency, and the persona that question represents
   • View SQL — Generated SQL for each question to see what the model produced

4. Read the per-question diagnostics, not just the aggregate score. Each failing question tells you something specific about what the model is missing.

5. Use the chat window to update your semantic model based on the failing questions. Describe the changes you want to make, and the chat will help you refine definitions, metrics, filters, and relationships.

6. Re-run the simulation and review the results. Check whether the questions you targeted now pass, and whether any previously passing questions regressed. Adjust and re-run until the results are stable.

   Databricks simulations

   On Databricks, simulations run sequentially due to Genie API rate limits (5 queries per minute per workspace by default). Expect longer run times than on Snowflake for the same question set. Databricks can raise this quota on request.

Once simulations consistently surface only out-of-scope questions and your domain expert is satisfied with the results, your context repository is ready to deploy.

Next steps

• Deploy to Snowflake: certify and push to Snowflake Cortex Analyst.
• Deploy to Databricks: certify and push to Databricks Genie.