## Set up cross-workspace extraction

URL: https://docs.atlan.com/apps/connectors/data-warehouses/databricks/how-tos/set-up-cross-workspace-extraction

> Configure a single service principal to crawl metadata from all workspaces within a Databricks metastore using system tables

Eliminate the need for separate crawler configurations by using a single service principal to crawl metadata from all workspaces within a Databricks metastore. This guide walks you through configuring the necessary permissions to enable cross-workspace extraction.

:::info Important!
Cross-workspace extraction isn't supported for REST API or JDBC extraction methods.
:::

## Prerequisites

Before you begin, make sure you have:

* A Unity Catalog-enabled Databricks workspace
* Account admin access to create and manage service principals
* Workspace admin access to grant permissions across all target workspaces
* At least one active SQL warehouse in each workspace you intend to crawl
* [Set up Databricks authentication](https://docs.atlan.com/llms/connectors/databricks/set-up-databricks/llms.txt) completed with one of the supported authentication methods
* [System table extraction enabled](https://docs.atlan.com/llms/connectors/databricks/set-up-databricks/llms.txt) for lineage and usage extraction

## Add service principal to all workspaces

You must use a **single, common service principal** that has been granted access to **all** Databricks workspaces you intend to crawl within the metastore.

1. Log in to your Databricks account console as an account admin
2. From the left menu, click **Workspaces** and select a workspace
3. From the tabs along the top, click the **Permissions** tab
4. In the upper right, click **Add permissions**
5. In the _Add permissions_ dialog:
 * For _User, group, or service principal_, select your service principal
 * For _Permission_, select workspace **User**
 * Click **Add**
6. Repeat steps 2-5 for each workspace you intend to crawl

## Permissions required

The service principal needs the following permissions on each workspace from which the you want Atlan to extract metadata and to enable cross-workspace extraction:

* `CAN_USE` on SQL warehouses in each workspace
* `USE CATALOG` on `system` catalog
* `USE SCHEMA` on `system.access` (for cross-workspace discovery)
* `USE SCHEMA` on `system.information_schema`
* `SELECT` on the following system tables:
 * `system.access.workspace_latest` (for cross-workspace discovery)
 * `system.information_schema.catalogs`
 * `system.information_schema.schemata`
 * `system.information_schema.tables`
 * `system.information_schema.columns`
 * `system.information_schema.key_column_usage`
 * `system.information_schema.table_constraints`
* `BROWSE` on all catalogs you want to crawl

## Grant permissions

Configure the necessary permissions for the service principal to access and extract metadata from all workspaces within the metastore.

1. **SQL workspace permissions:** The service principal must have usage permissions on **at least one active SQL warehouse within each workspace**. The extractor uses the smallest available warehouse to run its discovery queries.

### Via SQL

1. Connect to your Databricks workspace using a SQL client or the SQL editor
 2. Run the following command for each workspace, replacing the placeholders:

 ```sql
 GRANT CAN_USE ON WAREHOUSE <warehouse_name> TO `<service_principal_id>`;
 ```

 * Replace `<warehouse_name>` with your actual warehouse name
 * Replace `<service_principal_id>` with your service principal's application ID

 > *Example — see full content on the documentation site.*

### Via UI

1. Log in to your Databricks workspace as a workspace admin
 2. From the left menu, click **SQL Warehouses**
 3. On the _Compute_ page, for each SQL warehouse, click the 3-dot icon and then click **Permissions**
 4. In the _Manage permissions_ dialog:
 * In the _Type to add multiple users or groups_ field, search for and select your service principal
 * Select **Can use** permission
 * Click **Add** to assign the permission

2. **System table permissions:** Access to the system schema is essential for workspace and lineage discovery.

### Via SQL

1. Connect to your Databricks workspace using a SQL client or the SQL editor
 2. Grant system catalog access:

 ```sql
 GRANT USE CATALOG ON CATALOG system TO `<service_principal_id>`;
 ```

 3. Grant schema-level permissions:

 ```sql
 GRANT USE SCHEMA ON SCHEMA system.access TO `<service_principal_id>`;
 GRANT USE SCHEMA ON SCHEMA system.information_schema TO `<service_principal_id>`;
 ```

 4. Grant SELECT permissions on required system tables:

 ```sql
 -- For cross-workspace discovery
 GRANT SELECT ON TABLE system.access.workspace_latest TO `<service_principal_id>`;

 -- For metadata extraction
 GRANT SELECT ON TABLE system.information_schema.catalogs TO `<service_principal_id>`;
 GRANT SELECT ON TABLE system.information_schema.schemata TO `<service_principal_id>`;
 GRANT SELECT ON TABLE system.information_schema.tables TO `<service_principal_id>`;
 GRANT SELECT ON TABLE system.information_schema.columns TO `<service_principal_id>`;
 GRANT SELECT ON TABLE system.information_schema.key_column_usage TO `<service_principal_id>`;
 GRANT SELECT ON TABLE system.information_schema.table_constraints TO `<service_principal_id>`;
 ```

 * Replace `<service_principal_id>` with your service principal's application ID

 > *Example — see full content on the documentation site.*

### Via UI

1. Log in to your Databricks workspace as a workspace admin
 2. From the left menu, click **Catalog**
 3. In the _Catalog Explorer_, click on the **system** catalog
 4. Click the **Permissions** tab and then click **Grant**
 5. In the _Grant permissions_ dialog:
 * Under _Principals_, select your service principal
 * Under _Privileges_, check `USE CATALOG`
 * Click **Grant** to apply the permissions
 6. Navigate to **system** > **access**
 7. Click the **Permissions** tab and then click **Grant**
 8. In the _Grant permissions_ dialog:
 * Under _Principals_, select your service principal
 * Under _Privileges_, check `USE SCHEMA`
 * Click **Grant**
 9. Repeat for **system** > **information_schema**

 10. For each required system table, navigate to the table and grant `SELECT` permissions:
 * `system.access.workspace_latest`
 * `system.information_schema.catalogs`
 * `system.information_schema.schemata`
 * `system.information_schema.tables`
 * `system.information_schema.columns`
 * `system.information_schema.key_column_usage`
 * `system.information_schema.table_constraints`

3. **Asset permissions:** The service principal requires BROWSE permissions on all catalogs you want to crawl. BROWSE permission enables the service principal to see and read metadata for all data assets within the catalog, automatically granting access to all schemas and tables.

 :::info Important!
 For private catalogs, grant permissions from each workspace. For public catalogs, grant from any workspace. Only visible in the system tables when the service principal has BROWSE privileges on individual catalogs.
 :::

### Via SQL

1. Connect to your Databricks workspace using a SQL client or the SQL editor
 2. Grant BROWSE permissions on each catalog you want to crawl:

 ```sql
 GRANT BROWSE ON CATALOG <catalog_name> TO `<service_principal_id>`;
 ```

 * Replace `<catalog_name>` with your actual catalog name
 * Replace `<service_principal_id>` with your service principal's application ID

 > *Example — see full content on the documentation site.*

### Via UI

1. Log in to your Databricks workspace as a workspace admin
 2. From the left menu, click **Catalog**
 3. In the _Catalog Explorer_, navigate to the catalog you want to grant permissions on (for example, `main`)
 4. Click the **Permissions** tab and then click **Grant**
 5. In the _Grant permissions_ dialog:
 * Under _Principals_, select your service principal
 * Under _Privileges_, check **BROWSE**
 * Click **Grant** to apply the permissions
 6. Repeat steps 3-5 for each catalog you want to crawl in Atlan

## Need help?

- Check [Cross-workspace extraction setup FAQ](https://docs.atlan.com/llms/connectors/databricks/cross-workspace-extraction/llms.txt) for common questions about cross-workspace extraction
- Check [Troubleshooting cross-workspace extraction issues](https://docs.atlan.com/llms/connectors/databricks/cross-workspace-extraction-issues/llms.txt) for common issues
- [Contact Atlan support](https://docs.atlan.com/support/submit-request) for help with setup or integration

## Next steps

* [Crawl Databricks](https://docs.atlan.com/llms/connectors/databricks/crawl-databricks/llms.txt) - Set up and run a workflow to extract metadata from your Databricks instance using direct, offline, or agent extraction methods

---