Enable data quality on connection

Enable data quality on your Databricks connection in Atlan to start monitoring data quality.

Prerequisites

Before you enable Data Quality for Databricks, confirm that you have:

• Set up Databricks for data quality completed (including access request)
• Requested and received access for data quality from the Atlan team
• The service principal credentials created during Databricks setup
• Identify the Databricks connection where you want to enable data quality

Enable data quality

Once the Atlan team grants the access, follow these steps to enable data quality on your Databricks connection.

1. Turn on the data quality configuration toggle to configure:

   

   • Navigate to Admin in Atlan
   • Find the Labs section
   • Turn on the Databricks toggle under Data Quality section

2. Select your connection and configure credentials:

   Labs Page

   1. Navigate to Admin > Labs > Data Quality
   2. Click Connections for Databricks and select your connection from the list
   3. Click Configure for your selected connection
   4. Turn on Databricks data quality toggle
   5. Enter the following credential details:
      • Client ID: The service principal client ID created in Databricks setup
      • Client Secret: The service principal client secret
      • Tenant ID: The tenant ID (Azure only)
      • Workspace URL: Your Databricks workspace URL
      • SQL Warehouse: Your preferred SQL warehouse for DQ operations
   6. Enter your DQ catalog name: Atlan uses this catalog to store DQ related metadata for this connection in your Databricks environment.
   7. Click Run permissions check to verify:
      • Credentials have necessary permissions in Databricks
      • Databricks setup completed correctly
   8. Click Update to save the credentials

   Connection Settings

   1. Navigate to Governance > Connections
   2. Select your Databricks connection
   3. Open Connection settings from the sidebar
   4. Enter the following credential details:
      • Client ID: The service principal client ID created in Databricks setup
      • Client Secret: The service principal client secret
      • Tenant ID: The tenant ID (Azure only)
      • Workspace URL: Your Databricks workspace URL
      • SQL Warehouse: Your preferred SQL warehouse for DQ operations
   5. Enter your DQ catalog name: Atlan uses this catalog to store DQ related metadata for this connection in your Databricks environment.
   6. Click Run permissions check to verify:
      • Credentials have necessary permissions in Databricks
      • Databricks setup completed correctly
   7. Click Update to save the credentials

Next steps

After completing these steps:

• Atlan takes approximately 15 minutes to complete the setup in the background
• Once finished, you'll see data quality options available on your Databricks assets
• You can start creating data quality rules on tables and views

Need help

If you have questions or need assistance with enabling data quality on your connection, reach out to Atlan Support by submitting a support request.

See also

• Data quality permissions - Learn about the data quality permission scopes and configuration
• Configure alerts for data quality rules - Set up real-time notifications for rule failures