# Databricks

Integrate, catalog, and govern Databricks assets in Atlan.

## Required permissions

- [Preflight checks for Databricks](https://docs.atlan.com/llms/connectors/databricks/preflight-checks-for-databricks/llms.txt): Before running the Databricks crawler, you can run preflight checks to perform the necessary technical validations.
- [Permissions for Databricks AI models](https://docs.atlan.com/llms/connectors/databricks/permissions-for-databricks-ai-models/llms.txt): Full reference of the privileges required to crawl AI models and extract lineage from Databricks Unity Catalog, including what each privilege enables and how to grant it.
- [Permissions for Databricks AI/BI dashboards](https://docs.atlan.com/llms/connectors/databricks/permissions-for-databricks-ai-bi-dashboards/llms.txt): Full reference of the privileges required to crawl Databricks AI/BI dashboards and extract their lineage and usage metrics, including what each privilege enables and how to grant it.

## Setup options

- [How Atlan connects to Databricks](https://docs.atlan.com/llms/connectors/databricks/choose-how-atlan-connects-to-databricks/llms.txt): Understand the three connectivity paths between Atlan and Databricks—public, Private Link, and site-to-site VPN—and what determines which one applies to your workspace.
- [How to set up Databricks in Atlan](https://docs.atlan.com/llms/connectors/databricks/set-up-databricks/llms.txt): Set up Databricks for Atlan using one of three supported authentication methods—personal access token, service principal, or OAuth—to connect Atlan to your Databricks workspace.
- [Enable SSO for Databricks](https://docs.atlan.com/llms/connectors/databricks/enable-sso-for-databricks/llms.txt): Set up OAuth SSO authentication for Databricks connections in Atlan.
- [Set up cross-workspace extraction](https://docs.atlan.com/llms/connectors/databricks/set-up-cross-workspace-extraction/llms.txt): Configure a single service principal to crawl metadata from all workspaces within a Databricks metastore using system tables
- [Set up AWS private network link to Databricks](https://docs.atlan.com/llms/connectors/databricks/aws-private-link-to-databricks/llms.txt): Connect Atlan to an AWS-hosted Databricks workspace over AWS PrivateLink, so traffic never crosses the public internet.
- [Set up Azure private network link to Databricks](https://docs.atlan.com/llms/connectors/databricks/azure-private-link-to-databricks/llms.txt): Connect Atlan to an Azure Databricks workspace over Azure Private Link, including the endpoint approval handshake and the NSG rule.
- [Extract lineage and usage from Databricks](https://docs.atlan.com/llms/connectors/databricks/extract-lineage-and-usage-from-databricks/llms.txt): Retrieve lineage from Unity Catalog and usage and popularity metrics from query history or system tables using REST API, offline, or system table extraction methods.
- [Extract Databricks AI model lineage](https://docs.atlan.com/llms/connectors/databricks/extract-databricks-ai-model-lineage/llms.txt): Build upstream lineage between Databricks AI models and the datasets, tables, and functions they depend on.
- [Extract Databricks dashboard lineage](https://docs.atlan.com/llms/connectors/databricks/extract-databricks-dashboard-lineage/llms.txt): Build lineage from Databricks AI/BI dashboards to the Unity Catalog tables, columns, and metric views they read, and populate dashboard usage metrics.
- [Extract Databricks Genie Agent lineage](https://docs.atlan.com/llms/connectors/databricks/extract-databricks-genie-lineage/llms.txt): Build lineage from Databricks Genie Agents to the Unity Catalog tables and columns they read.
- [Manage Databricks tags](https://docs.atlan.com/llms/connectors/databricks/manage-databricks-tags/llms.txt): You must have a Unity Catalog-enabled workspace and SQL warehouse configured to import Databricks tags in Atlan.
- [Push description updates to Databricks](https://docs.atlan.com/llms/connectors/databricks/push-description-updates-to-databricks/llms.txt): Enable reverse sync to push asset descriptions authored in Atlan back to Databricks as comments.
- [Grant query text access for Databricks](https://docs.atlan.com/llms/connectors/databricks/grant-query-text-access/llms.txt): Add Atlan's service principal to the databricks_pii_access group to restore query text after Databricks masks it by default.
- [Set up on-premises Databricks access](https://docs.atlan.com/llms/connectors/databricks/set-up-on-premises-databricks-access/llms.txt): The Docker-based databricks-extractor offline tool has been sunset. For on-premises or network-restricted Databricks environments, use Self-Deployed Runtime, Secure Agent, or direct connectivity via private link.
- [Crawl on-premises Databricks](https://docs.atlan.com/llms/connectors/databricks/crawl-on-premises-databricks/llms.txt): The Docker-based databricks-extractor offline tool has been sunset. For on-premises or network-restricted Databricks environments, use Self-Deployed Runtime, Secure Agent, or direct connectivity via private link.
- [Set up on-premises Databricks lineage extraction](https://docs.atlan.com/llms/connectors/databricks/set-up-on-premises-databricks-lineage-extraction/llms.txt): The Docker-based databricks-extractor offline tool has been sunset. For on-premises or network-restricted Databricks lineage extraction, use Self-Deployed Runtime, Secure Agent, or direct connectivity via private link.
- [Extract on-premises Databricks lineage](https://docs.atlan.com/llms/connectors/databricks/extract-on-premises-databricks-lineage/llms.txt): The Docker-based databricks-extractor offline tool has been sunset. For on-premises or network-restricted Databricks lineage extraction, use Self-Deployed Runtime, Secure Agent, or direct connectivity via private link.

## Crawl and mine

- [How to crawl Databricks in Atlan](https://docs.atlan.com/llms/connectors/databricks/crawl-databricks/llms.txt): Crawl metadata from Databricks in Atlan to catalog tables, schemas, views, and volumes. Configure access permissions and authentication before extracting lineage and column information.
- [Crawl Databricks AI models](https://docs.atlan.com/llms/connectors/databricks/crawl-databricks-ai-models/llms.txt): Discover and catalog AI models registered in the Databricks Unity Catalog Model Registry using Atlan's Databricks connector.
- [Crawl Databricks AI/BI dashboards](https://docs.atlan.com/llms/connectors/databricks/crawl-databricks-ai-bi-dashboards/llms.txt): Discover and catalog Databricks AI/BI (Lakeview) dashboards using Atlan's Databricks connector.
- [Crawl Databricks Genie Agents](https://docs.atlan.com/llms/connectors/databricks/crawl-databricks-genie-agents/llms.txt): Discover and catalog Databricks Genie spaces as Genie Agent assets using Atlan's Databricks connector.

## Reference

- [How private connectivity works](https://docs.atlan.com/llms/connectors/databricks/how-private-connectivity-works/llms.txt): A plain-language explanation of what a private link to Databricks is, who builds which half, and the three traffic paths between Atlan and Databricks.
- [Lineage freshness strategies](https://docs.atlan.com/llms/connectors/databricks/lineage-freshness-strategies/llms.txt): Understand how Atlan filters Databricks lineage data to show only valid lineage relationships.
- [What does Atlan crawl from Databricks?](https://docs.atlan.com/llms/connectors/databricks/what-does-atlan-crawl-from-databricks/llms.txt): During a Databricks crawl, Atlan extracts metadata for Databases, Schemas, Tables, Views, Materialized Views, Volumes, External Locations, and AI Models. Reference tables map each Databricks property to its Atlan asset.
- [Databricks app image and versioning](https://docs.atlan.com/llms/connectors/databricks/databricks-app-image-and-versioning/llms.txt): Identify the container image used for the Databricks assets app, understand its versioning, and find the exact version assigned to your tenant.
- [Allowing Atlan IPs in Databricks](https://docs.atlan.com/llms/connectors/databricks/allowlist-atlan-ips-in-databricks/llms.txt): Add Atlan's addresses to your Databricks workspace IP access list (or a corporate firewall in the path) so crawlers and miners can connect.
- [Databricks assets package](https://docs.atlan.com/llms/connectors/databricks/package-reference/llms.txt): Programmatically crawl and manage Databricks assets in Atlan using the Python SDK (pyatlan). Reference for the Databricks assets package.
- [Databricks assets app](https://docs.atlan.com/llms/connectors/databricks/databricks-sdk-references-app-reference/llms.txt): Learn how to crawl Databricks assets and publish them to Atlan for discovery.
- [Databricks miner app](https://docs.atlan.com/llms/connectors/databricks/databricks-sdk-references-app-miner-reference/llms.txt): Learn how to mine query history from Databricks to generate lineage and usage metrics.
- [Databricks miner package](https://docs.atlan.com/llms/connectors/databricks/miner-reference/llms.txt): Programmatically extract lineage and usage from Databricks in Atlan using the Python SDK (pyatlan).

## Troubleshooting

- [Databricks connectivity](https://docs.atlan.com/llms/connectors/databricks/troubleshooting-databricks-connectivity/llms.txt): Troubleshoot Databricks connectivity, covering network and private-link issues, authentication, permissions, extraction, and metadata questions.
- [Cross-workspace extraction issues](https://docs.atlan.com/llms/connectors/databricks/cross-workspace-extraction-issues/llms.txt): Troubleshoot common issues in Databricks cross-workspace extraction with error, cause, and solution guidance.
- [Special cases in Databricks connectivity](https://docs.atlan.com/llms/connectors/databricks/databricks-connectivity-special-cases/llms.txt): Multiple workspaces, extra hostnames and custom ports, cross-cloud and cross-region setups, site-to-site VPNs, and the reversed traffic of lakehouse and MCP integrations.
- [Cross-workspace extraction](https://docs.atlan.com/llms/connectors/databricks/cross-workspace-extraction/llms.txt): Frequently asked questions about Databricks cross-workspace extraction setup and configuration.
- [Atlan vs Databricks lineage](https://docs.atlan.com/llms/connectors/databricks/atlan-vs-databricks-lineage/llms.txt): Frequently asked questions about how Atlan lineage differs from Databricks native lineage.
- [How Atlan builds lineage for Databricks AI models](https://docs.atlan.com/llms/connectors/databricks/how-databricks-ai-model-lineage-works/llms.txt): Frequently asked questions about how Atlan constructs lineage for Databricks AI models, including lineage sources, relationships created, and asset behavior.
- [How Atlan builds lineage for Databricks dashboards](https://docs.atlan.com/llms/connectors/databricks/how-databricks-dashboard-lineage-works/llms.txt): Frequently asked questions about how Atlan constructs lineage for Databricks AI/BI dashboards, including lineage sources, relationships created, and known limits.
- [How Atlan builds lineage for Databricks Genie Agents](https://docs.atlan.com/llms/connectors/databricks/how-databricks-genie-lineage-works/llms.txt): Frequently asked questions about how Atlan catalogs Databricks Genie spaces and constructs their lineage, including lineage sources, relationships created, and known limits.
- [Query text access in Databricks](https://docs.atlan.com/llms/connectors/databricks/databricks-sql-masking-change/llms.txt): What Databricks's SQL query text masking change affects in Atlan, what it doesn't affect, and how to keep query-text features running.
- [Description reverse sync delays](https://docs.atlan.com/llms/connectors/databricks/description-reverse-sync-delays/llms.txt): Frequently asked questions about timing and delays when pushing Databricks description updates from Atlan.
- [Unity Catalog governance and connections](https://docs.atlan.com/llms/connectors/databricks/unity-catalog-governance/llms.txt): Frequently asked questions about Unity Catalog connection scope, cross-workspace access, and which governance objects Atlan imports from Databricks.