Skip to main content

Crawl Qualytics

Configure and run the Qualytics crawler to extract data quality metadata and enrich your existing Atlan assets with comprehensive quality insights. The crawler maps your Atlan connections to Qualytics data sources and syncs quality scores, check counts, anomaly information, and direct links to Qualytics.

Prerequisites​

Before you begin, make sure you have:

  • Completed Qualytics setup
  • Admin or connection admin privileges in Atlan
  • At least one connection configured in Atlan that corresponds to a data source in Qualytics
  • Qualytics Host URL (for example, https://your-company.qualytics.io)
  • Qualytics API token (generate from Qualytics under Settings > API Tokens)
  • Atlan API token (create from Atlan under Settings > API Tokens)

Create crawler workflow​

Follow these steps to create a workflow in Atlan that captures data quality metadata from Qualytics.

  1. In Atlan, select New > New Workflow.

  2. From the package list, choose Qualytics.

  3. Select Setup Workflow.

Configure authentication​

You need to provide your Qualytics API credentials to authenticate the connection.

  1. Enter the base URL of your Qualytics instance in Qualytics Host.

  2. Enter your Qualytics API token in Qualytics API Token.

  3. Enter your Atlan API token in Atlan API Token.

  4. Click Test Authentication to verify connectivity to Qualytics.

  5. After authentication succeeds, click Next at the bottom of the screen.

Configure connection​

To complete the Qualytics connection configuration:

  1. Provide a Connection Name that represents your source environment. For example, you might use values like production, development, or analytics.
  2. By default, all admins can manage the connection. If you want to restrict access, update the users or groups listed under Connection Admins. If you remove all entries, no one can manage the connection.
  3. If you want the crawler to target a specific data source, select the matching connection from Connections Filter that corresponds to your Qualytics data source. This helps the crawler enrich the correct assets with quality metadata from Qualytics and can improve workflow performance by limiting the search scope. Otherwise, the crawler searches across all available connections, which may take longer in larger deployments.
  4. At the bottom of the screen, click Next to proceed.

Run crawler​

To run the Qualytics crawler, after completing the previous steps:

  1. Review your configuration to confirm the settings are correct.

  2. You can either:

    • To run the crawler once immediately, at the bottom of the screen, click the Run button.
    • To schedule the crawler to run hourly, daily, weekly, or monthly, at the bottom of the screen, click the Schedule & Run button.

Once the crawler has completed running, your existing Atlan assets are enriched with data quality metadata from Qualytics. This includes quality scores across eight dimensions, check counts, anomaly information, and direct links to Qualytics that appear on your table and column assets.

See also​