Skip to main content

Crawl Hive

Extract metadata assets from your Hive database into Atlan.

Prerequisites​

Before you begin, verify you have:

Create crawler workflow​

Create a new workflow and select Hive as your connector source.

  1. In the top navigation, click Marketplace.
  2. Search for Hive Assets and select it.
  3. Click Install.
  4. Once installation completes, click Setup Workflow on the same tile.

If you navigated away before installation completed, go to New > New Workflow and select Hive Assets to proceed.

Configure extraction​

When setting up metadata extraction from your Hive database, choose how Atlan connects and extracts metadata. Select the extraction method that best fits your organization's security and network requirements:

Offline extraction sunset

The offline extraction method has been sunset and is no longer available. For on-premises or network-restricted environments, use the Agent extraction method with Self-Deployed Runtime.

Atlan SaaS connects directly to your Hive database (HiveServer2). This method supports Basic and Kerberos authentication and lets you test the connection before proceeding.

  1. Choose an authentication type that matches your Hive configuration.
  1. Select the Basic authentication type.
  2. For Host Name, enter the host name (or PrivateLink endpoint) for your Hive instance.
  3. For Port, enter the port number for your Hive instance (default: 10000).
  4. For Username, enter the username you created for that instance.
  5. For Password, enter the password for the username.
  6. For Default Schema, enter the default schema name for your Hive instance.
  1. Select the security type:

No additional configuration required. The connection isn't encrypted. Use only in trusted internal networks.

  1. Click Test Authentication to confirm connectivity to Hive. When successful, click Next to proceed with the connection configuration.

Configure connection​

Set up the connection name and access controls for your Hive data source in Atlan.

  1. Provide a Connection Name that represents your source environment. For example, you might use values like production, development, gold, or analytics.
  2. To change the users able to manage this connection, update the users or groups listed under Connection Admins. If you don't specify any user or group, nobody can manage the connection (not even admins).
  3. At the bottom of the screen, click Next to proceed.

Configure crawler​

Before running the crawler, you can configure which assets to include or exclude. On the Metadata page:

  • To exclude specific assets from crawling, click Exclude Metadata. This defaults to no assets if none are specified.
  • To include specific assets in crawling, click Include Metadata. This defaults to all assets if none are specified.

If an asset appears in both the include and exclude filters, the exclude filter takes precedence.

Run crawler​

  1. Click Preflight checks to validate permissions and configuration before running the crawler. This helps identify any potential issues early.
  2. After the preflight checks pass, you can either:
    • Click Run to run the crawler once immediately.
    • Click Schedule Run to schedule the crawler to run hourly, daily, weekly, or monthly.

Once the crawler has completed running, you can see the assets on Atlan's asset page.

Did you know?

Once you have crawled assets from Hive, you can run the Hive Miner to mine query history through S3.

See also​