## Crawl Databricks AI models

URL: https://docs.atlan.com/apps/connectors/data-warehouses/databricks/how-tos/crawl-databricks-ai-models

> Discover and catalog AI models registered in the Databricks Unity Catalog Model Registry using Atlan's Databricks connector.

Atlan can discover and catalog AI models—and their logged versions—registered in the [Databricks Unity Catalog Model Registry](https://docs.databricks.com/en/machine-learning/manage-model-lifecycle/index.html). Once crawled, model assets are visible in Atlan alongside your other Databricks data assets. Model crawling requires the **Direct** extraction strategy and isn't supported with the Offline or Agent extraction strategies.

## Modeling Databricks Genies

Databricks [Genie](https://docs.databricks.com/aws/en/genie/) spaces are end-user, conversational AI experiences built on top of one or more underlying models. In Atlan, model them as **AI Applications**, not AI Models:

- **AI Models** represent the underlying model and its versions (the trained artifact and its lifecycle).
- **AI Applications** represent the end-user, AI-powered experience built on top of one or more models—which is what a Genie space is.

To bring a Genie into Atlan, [create an AI app](https://docs.atlan.com/product/capabilities/governance/ai-governance/get-started/create-ai-app) and link the underlying Databricks AI models and input datasets the Genie consumes as its inputs.

## Prerequisites

Before crawling AI models, make sure you have:

- A [Unity Catalog-enabled](https://docs.databricks.com/en/data-governance/unity-catalog/get-started.html) Databricks workspace
- [Set up the Databricks connector](https://docs.atlan.com/llms/connectors/databricks/set-up-databricks/llms.txt) in Atlan
- [Crawled Databricks assets](https://docs.atlan.com/llms/connectors/databricks/crawl-databricks/llms.txt) at least once

## Permissions required

In addition to the standard [Databricks connector permissions](https://docs.atlan.com/llms/connectors/databricks/set-up-databricks/llms.txt), the Atlan service account requires:

- **Data Reader** preset (or the individual privileges `USE CATALOG`, `USE SCHEMA`, `EXECUTE`, `READ VOLUME`, and `SELECT`) on all catalogs and schemas containing models
- **CAN VIEW** or **CAN READ** on all user notebooks and MLflow experiments linked to model versions. To cover all model versions without granting access notebook by notebook, grant **CAN VIEW** at the workspace level.
- `READ FILES` on any external location storing `feature_spec.yaml` artifacts, if your workspace uses [Databricks Feature Store](https://docs.databricks.com/en/machine-learning/feature-store/index.html) models with externally stored artifacts:

```sql
GRANT READ FILES ON EXTERNAL LOCATION <external_location_name> TO <atlan_user_or_role>;
```

For the full breakdown of each privilege and what it enables, see [Permissions for Databricks AI models](https://docs.atlan.com/llms/connectors/databricks/permissions-for-databricks-ai-models/llms.txt).

## Configure crawler

To configure the crawler for AI models:

1. Follow the standard [Crawl Databricks](https://docs.atlan.com/llms/connectors/databricks/crawl-databricks/llms.txt) steps.
2. When selecting the extraction strategy, choose **Direct**.
3. For the extraction method, select **System Tables**. The **REST API** method is deprecated—use System Tables instead. System Tables supports all authentication types: personal access token, AWS service principal, and Azure service principal.
4. Under asset filters, specify which catalogs or schemas to crawl:
 - To include specific catalogs or schemas, click **Include Metadata**.
 - To exclude specific catalogs or schemas, click **Exclude Metadata**.
 - If no filters are set, Atlan crawls all catalogs and schemas accessible to the service account.
5. Run the workflow.

After the workflow completes, **AI Model** and **AI Model Version** assets appear in Atlan under the crawled catalog and schema.

## See also

- [Extract Databricks AI model lineage](https://docs.atlan.com/llms/connectors/databricks/extract-databricks-ai-model-lineage/llms.txt): Build lineage between your Databricks models and the upstream data assets they depend on.
- [What does Atlan crawl from Databricks](https://docs.atlan.com/llms/connectors/databricks/what-does-atlan-crawl-from-databricks/llms.txt): Full reference of all Databricks assets and properties crawled by Atlan, including AI models.

---