## Crawl Hive

URL: https://docs.atlan.com/apps/connectors/database/hive/how-tos/crawl-hive

> Extract metadata from Hive to catalog databases, tables, views, and columns. Extract lineage and asset information after configuring user permissions.

Extract metadata assets from your Hive database into Atlan.

## Prerequisites

Before you begin, verify you have:

* [Configured the Hive permissions](https://docs.atlan.com/llms/connectors/hive/set-up-hive/llms.txt) (and [set up a private network link](https://docs.atlan.com/llms/connectors/hive/set-up-a-private-network-link-to-hive/llms.txt) if using private network)
* Access to your Hive instance and credentials
* Reviewed the [order of operations](https://docs.atlan.com/llms/catalog/connector-capabilities/order-workflows/llms.txt)

## Create crawler workflow

Create a new workflow and select Hive as your connector source.

1. In your Atlan workspace, click **Connectors** in the left sidebar.
 * If you are using the **Old UI (Classic)**, click **New Workflow** in the top navigation.

2. Click **Marketplace**.

3. Search for **Hive Assets** and select it.

4. Click **Install**.

5. Once installation completes, click **Setup Workflow** on the same tile.

### Configure extraction

When setting up metadata extraction from your Hive database, choose how Atlan connects and extracts metadata. Select the extraction method that best fits your organization's security and network requirements:

:::warning Offline extraction sunset
The offline extraction method has been sunset and is no longer available. For on-premises or network-restricted environments, use the **Agent** extraction method with [Self-Deployed Runtime](https://docs.atlan.com/llms/platform/self-deployed-runtime/llms.txt).
:::

### Direct

Atlan SaaS connects directly to your Hive database (HiveServer2). This method supports Basic and Kerberos authentication and lets you test the connection before proceeding.

1. Choose an authentication type that matches your Hive configuration.

1. Select the **Basic** authentication type.

2. For **Host Name**, enter the host name (or [PrivateLink endpoint](https://docs.atlan.com/llms/connectors/hive/set-up-a-private-network-link-to-hive/llms.txt)) for your Hive instance.

3. For **Port**, enter the port number for your Hive instance (default: 10000).

4. For **Username**, enter the username you created for that instance.

5. For **Password**, enter the password for the username.

6. For **Default Schema**, enter the default schema name for your Hive instance.

### Kerberos authentication

1. Select the **Kerberos** authentication type.

2. For **Host Name**, enter the host name (or [PrivateLink endpoint](https://docs.atlan.com/llms/connectors/hive/set-up-a-private-network-link-to-hive/llms.txt)) for your Hive instance.

3. For **Port**, enter the port number for your Hive instance (default: 10000).

4. For **Kerberos Principal**, enter your user principal in the format `username@REALM` (for example, `atlan_user@ATLAN.LOCAL`).

5. For **Service Name**, enter the Hive service principal name (typically `hive`).

6. For **Keytab File**, upload the keytab file you [generated during setup](https://docs.atlan.com/llms/connectors/hive/set-up-hive/llms.txt).

7. For **krb5.conf File**, upload your [Kerberos configuration file](https://docs.atlan.com/llms/connectors/hive/set-up-hive/llms.txt).

8. For **Default Schema**, enter the default schema name for your Hive instance.

2. Select the security type:

### Default (No TLS)

No additional configuration required. The connection isn't encrypted. Use only in trusted internal networks.

### TLS

1. Select **TLS** as the security type.

2. For **CA Certificate File**, upload the CA certificate that signed your HiveServer2's SSL certificate (PEM, CRT, or `.zip` archive).

### mTLS

1. Select **MTLS** as the security type.

2. For **CA Certificate File**, upload the CA certificate that signed your HiveServer2's SSL certificate.

3. For **Client Certificate File**, upload your client certificate.

4. For **Client Private Key File**, upload your client private key.

5. For **Client Key Passphrase**, enter the passphrase if your client key is encrypted (optional).

6. All certificates must be in PEM or CRT format, or uploaded as `.zip` archives.

3. Click **Test Authentication** to confirm connectivity to Hive. When successful, click **Next** to proceed with the connection configuration.

### Agent

Self-Deployed Runtime runs within your organization and connects to your Hive database. This method keeps connections inside your network perimeter.

1. Install Self-Deployed Runtime if you haven't already:
 - [Install via Docker Compose](https://docs.atlan.com/llms/platform/self-deployed-runtime/install-on-docker-compose/llms.txt)
 - [Install on Kubernetes](https://docs.atlan.com/llms/platform/self-deployed-runtime/install-on-kubernetes/llms.txt)

2. Select the **Agent** tab and configure the Hive data source by adding the secret keys for your secret store.

3. Choose an authentication type that matches your Hive configuration:

1. Select the **Basic** authentication type.

2. For **Host Name**, enter the host name (or [PrivateLink endpoint](https://docs.atlan.com/llms/connectors/hive/set-up-a-private-network-link-to-hive/llms.txt)) for your Hive instance.

3. For **Port**, enter the port number for your Hive instance (default: 10000).

4. For **Username**, enter the username you created for that instance (or reference the secret key where it's stored).

5. For **Password**, enter the password for the username (or reference the secret key where it's stored).

6. For **Default Schema**, enter the default schema name for your Hive instance.

### Kerberos authentication

1. Select the **Kerberos** authentication type.

2. For **Host Name**, enter the host name (or [PrivateLink endpoint](https://docs.atlan.com/llms/connectors/hive/set-up-a-private-network-link-to-hive/llms.txt)) for your Hive instance.

3. For **Port**, enter the port number for your Hive instance (default: 10000).

4. For **Kerberos Principal**, enter your user principal in the format `username@REALM` (for example, `atlan_user@ATLAN.LOCAL`).

5. For **Service Name**, enter the Hive service principal name (typically `hive`).

6. For **Keytab File**, provide the keytab file you [generated during setup](https://docs.atlan.com/llms/connectors/hive/set-up-hive/llms.txt) using any of the [input forms supported in SDR mode](https://docs.atlan.com/llms/platform/self-deployed-runtime/configure-workflow-execution/llms.txt).

7. For **Kerberos Config File (krb5.conf)**, provide your [Kerberos configuration file](https://docs.atlan.com/llms/connectors/hive/set-up-hive/llms.txt) using any of the same forms.

8. For **Default Schema**, enter the default schema name for your Hive instance.

4. Select the security type:

### Default (No TLS)

No additional configuration required. The connection isn't encrypted. Use only in trusted internal networks.

### TLS

1. Select **TLS** as the security type.

2. For **CA Certificate File**, provide the CA certificate that signed your HiveServer2's SSL certificate (PEM, CRT, or `.zip` archive) using any of the [input forms supported in SDR mode](https://docs.atlan.com/llms/platform/self-deployed-runtime/configure-workflow-execution/llms.txt).

### mTLS

1. Select **MTLS** as the security type.

2. For **CA Certificate File**, provide the CA certificate that signed your HiveServer2's SSL certificate using any of the [input forms supported in SDR mode](https://docs.atlan.com/llms/platform/self-deployed-runtime/configure-workflow-execution/llms.txt).

3. For **Client Certificate File**, provide your client certificate using any of the same forms.

4. For **Client Private Key File**, provide your client private key using any of the same forms.

5. For **Client Key Passphrase**, enter the passphrase if your client key is encrypted, or reference the secret key (optional).

6. All certificates must be in PEM or CRT format, or uploaded as `.zip` archives.

5. Complete the configuration by following [Configure workflow for SDR mode of execution](https://docs.atlan.com/llms/platform/self-deployed-runtime/configure-workflow-execution/llms.txt).

6. Click **Next** after completing the configuration.

### Configure connection

Set up the connection name and access controls for your Hive data source in Atlan.

1. Provide a **Connection Name** that represents your source environment. For example, you might use values like `production`, `development`, `gold`, or `analytics`.

2. To change the users able to manage this connection, update the users or groups listed under **Connection Admins**. If you don't specify any user or group, nobody can manage the connection (not even admins).

3. At the bottom of the screen, click **Next** to proceed.

### Configure crawler

Before running the crawler, you can configure which assets to include or exclude. On the **Metadata** page:

- To exclude specific assets from crawling, click **Exclude Metadata**. This defaults to no assets if none are specified.
- To include specific assets in crawling, click **Include Metadata**. This defaults to all assets if none are specified.

If an asset appears in both the include and exclude filters, the exclude filter takes precedence.

### Run crawler

### Direct

1. Click **Preflight checks** to validate permissions and configuration before running the crawler. This helps identify any [potential issues](https://docs.atlan.com/llms/connectors/hive/preflight-checks-for-hive/llms.txt) early.

2. After the preflight checks pass, you can either:
 - Click **Run** to run the crawler once immediately.
 - Click **Schedule Run** to schedule the crawler to run hourly, daily, weekly, or monthly.

### Agent

You can either:

- Click **Run** to run the crawler once immediately.
- Click **Schedule Run** to schedule the crawler to run hourly, daily, weekly, or monthly.

Once the crawler has completed running, you can see the assets on Atlan's asset page.

:::info **Did you know?**
Once you have crawled assets from Hive, you can run the **Hive Miner** to [mine query history through S3](https://docs.atlan.com/llms/catalog/connector-capabilities/mine-queries-through-cloud-object-storage/llms.txt).
:::

## See also

- [How Atlan connects to Hive](https://docs.atlan.com/llms/connectors/hive/how-atlan-connects-to-hive/llms.txt): Connectivity, authentication, and data access patterns
- [What does Atlan crawl from Hive](https://docs.atlan.com/llms/connectors/hive/what-does-atlan-crawl-from-hive/llms.txt): Metadata and assets discovered during crawling
- [Preflight checks for Hive](https://docs.atlan.com/llms/connectors/hive/preflight-checks-for-hive/llms.txt): Verify prerequisites before crawling
- [Troubleshooting Hive connectivity](https://docs.atlan.com/llms/connectors/hive/troubleshooting-hive-connectivity/llms.txt): Resolve common connection issues

---