Enable data quality on connection Private preview
Enable data quality on your Databricks connection in Atlan to start monitoring data quality. This guide helps you configure the connection with the necessary credentials and permissions.
Prerequisites
Before you begin, complete the following steps:
- Set up Databricks for data quality completed
- Have the service principal credentials created during Databricks setup
- Identify the Databricks connection where you want to enable data quality
Enable data quality
Follow these steps to enable data quality on your Databricks connection.
-
Turn on the data quality feature:
- Navigate to Settings in Atlan
- Find the Labs section
- Turn on the Data Quality toggle
-
Select your connection and configure credentials:
IMPORTANTCurrently, you can only enable data quality on one connection in Atlan. If you wish to enable it on another connection, raise a support request.
- Data Quality Page
- Connection Settings
- Navigate to Governance > Data Quality
- Select your Databricks connection from the list
- Click Enable data quality for your selected connection
- Enter the following credential details:
- Client ID: The service principal client ID created in Databricks setup
- Client Secret: The service principal client secret
- Tenant ID: The tenant ID (Azure only)
- Workspace URL: Your Databricks workspace URL
- SQL Warehouse: Your preferred SQL warehouse for DQ operations
- Click Run permissions check to verify:
- Credentials have necessary permissions in Databricks
- Databricks setup completed correctly
- Click Update to save the credentials
- Navigate to Governance > Connections
- Select your Databricks connection
- Open Connection settings from the sidebar
- Enter the following credential details:
- Client ID: The service principal client ID created in Databricks setup
- Client Secret: The service principal client secret
- Tenant ID: The tenant ID (Azure only)
- Workspace URL: Your Databricks workspace URL
- SQL Warehouse: Your preferred SQL warehouse for DQ operations
- Click Run permissions check to verify:
- Credentials have necessary permissions in Databricks
- Databricks setup completed correctly
- Click Update to save the credentials
Next steps
After completing these steps:
- Atlan takes approximately 10 minutes to complete the setup in the background
- Once finished, you'll see data quality options available on your Databricks assets
- You can start creating data quality rules on tables and views
Need help
If you have questions or need assistance with enabling data quality on your connection, reach out to Atlan Support by submitting a support request.
See also
- Data quality permissions - Learn about the data quality permission scopes and configuration
- Configure alerts for data quality rules - Set up real-time notifications for rule failures