29 docs tagged with "databricks"

Frequently asked questions about how Atlan lineage differs from Databricks native lineage.

Understand how Data Quality Studio uses compute resources, how costs are calculated, and practical ways to track and optimize spend for Snowflake, Databricks, and BigQuery.

Configure BYOC credentials and query settings on your BigQuery, Databricks, or Snowflake connection to enable failed rows queries.

Connect Databricks to Atlan's Lakehouse using foreign Iceberg tables in Unity Catalog to query metadata. Setup creates storage credentials and an external location pointing to Atlan's Lakehouse data in your cloud storage.

Discover and catalog AI models registered in the Databricks Unity Catalog Model Registry using Atlan's Databricks connector.

Frequently asked questions about Databricks cross-workspace extraction setup and configuration.

Troubleshoot common issues in Databricks cross-workspace extraction with error, cause, and solution guidance.

Monitor and maintain data quality across your data sources with automated quality checks, alerts, and governance workflows

Use Lakehouse to analyze database usage, optimize query performance, and manage storage and compute costs

Integrate, catalog, and govern Databricks assets in Atlan.

Set up and configure Databricks for data quality monitoring through Atlan.

Deploy a context repository in Context Engineering Studio to Databricks Genie. CES pushes descriptions into Unity Catalog, creates one Metric View per table, and configures a Genie Space. On Databricks, deploying is part of the build cycle because Chat and build and Simulate run on the live Genie Space.

Enable and configure data quality for your Databricks connection in Atlan.

Configure data quality rules to scan only the most recent day's data for faster, more efficient monitoring.

Set up OAuth SSO authentication for Databricks connections in Atlan.

Build upstream lineage between Databricks AI models and the datasets, tables, and functions they depend on.

Grant the Unity Catalog and workspace permissions Context Engineering Studio needs to deploy a context repository to Databricks Genie. Required at deploy time. On Databricks, Simulate also runs on a live Genie Space, so simulating requires a first deploy.

Frequently asked questions about how Atlan constructs lineage for Databricks AI models, including lineage sources, relationships created, and asset behavior.

Understand how Atlan filters Databricks lineage data to show only valid lineage relationships.

Full reference of the privileges required to crawl AI models and extract lineage from Databricks Unity Catalog, including what each privilege enables and how to grant it.

View and export the actual data rows that failed data quality rules to investigate and resolve data quality issues.

Reference for restricting data quality rules to only the most recent day's data.

Trigger data quality rules immediately at the table or rule level without waiting for the next scheduled run. Supported for Snowflake and Databricks.

Configure a single service principal to crawl metadata from all workspaces within a Databricks metastore using system tables

Configure Databricks to enable data quality monitoring through Atlan.

Common questions about Databricks data quality setup and configuration.

Resolve common Databricks errors in Context Engineering Studio, including OAuth token failures, Genie Space creation issues, Unity Catalog access errors, Metric View YAML validation, SQL warehouse problems, and chat errors.

Resolve common Databricks errors when creating and querying foreign Iceberg tables in Unity Catalog.

Let Atlan suggest data quality rules automatically based on your asset's metadata structure, and apply them in a few clicks.