Skip to main content

Setup and configuration

This document answers common questions about prerequisites, permissions, and environment settings required to run Atlan’s Data Quality Studio on Databricks.

What Databricks edition is required for data quality?​

Atlan DQ support for Databricks is supported only on Premium and Enterprise tiers of Databricks.

What administrative access is required?​

The user performing the setup must be:

  • A Workspace admin; and
  • A Metastore Admin or have CREATE CATALOG privilege on the metastore linked to the workspace

Is serverless compute required?​

Yes, your workspace must have the following feature enabled:

  • Serverless Compute for Jobs & Notebooks

This is required to permit execution of Atlan's DQ jobs in your Databricks Workspace using Serverless compute.

A dedicated SQL warehouse must be identified for running DQ-related queries. While Atlan supports any SQL Warehouse, Atlan recommends using a Serverless SQL Warehouse for faster startup times.

Is network access configuration required?​

Outbound Network Access Must Be Allowed from Serverless Compute: Databricks Serverless Compute uses network policies to control outbound traffic [only for Enterprise tier]. Verify that outbound connectivity to Atlan is permitted from the Serverless environment.

What Atlan prerequisites are needed?​

Before integrating with Databricks, you need to generate an API token in Atlan. This token is securely stored in Databricks in a secret and used to authenticate API requests from within Databricks.

Can I enable data quality on multiple connections?​

Currently, you can only enable data quality on one connection in Atlan. If you wish to enable it on another connection, raise a support request.

How long does the setup take?​

After completing the setup steps, Atlan takes approximately 10 minutes to complete the setup in the background. Once finished, you'll see data quality options available on your Databricks assets.

Can I use private channels for alerts?​

Only public channels are supported for data quality alerts. Alerts can't be routed to private channels or Direct Messages at this time.