## Crawl CrateDB

URL: https://docs.atlan.com/apps/connectors/database/cratedb/how-tos/crawl-cratedb

> Configure and run the CrateDB crawler to extract metadata from your database

Extract metadata from your CrateDB database and make it available in Atlan for data discovery, governance, and lineage tracking. This guide walks you through setting up authentication and running your first crawl.

## Prerequisites

Before you begin, make sure you have:
* [Set up CrateDB](https://docs.atlan.com/llms/connectors/cratedb/set-up-cratedb/llms.txt) with proper user permissions
* Network connectivity between Atlan and your CrateDB instance
* Your CrateDB cluster HTTP endpoint and port information

## Set up workflow

Create a new **CrateDB Assets** workflow to extract metadata from your database.

1. In your Atlan workspace, click **Connectors** in the left sidebar.
 * If you are using the **Old UI (Classic)**, click **New Workflow** in the top navigation.

2. Click **Marketplace**.

3. Search for **CrateDB Assets** and select it.

4. Click **Install**.

5. Once installation completes, click **Setup Workflow** on the same tile.

### Configure extraction method

Choose how to connect to your CrateDB environment:

### Direct extraction

1. Select **Direct** for the extraction method.

2. Enter your CrateDB connection details:
 - **Host**: Your CrateDB cluster HTTP endpoint (for example, `https://your-cluster.crate.io`)
 - **Port**: The port number of your CrateDB instance
 - **Authentication**: Choose **Basic** authentication
 - **Username**: Enter the username you configured in CrateDB
 - **Password**: Enter the password you configured in CrateDB
 - **Database**: Enter the name of the database to crawl
3. Click **Test Authentication** to confirm connectivity to CrateDB using these details.

4. When successful, click **Next**.

### Agent extraction

1. Select **Agent** for the extraction method.

2. Add the secret keys for your secret store configuration.

3. Follow the [Secure Agent configuration guide](https://docs.atlan.com/llms/platform/secure-agent/configure-secure-agent-for-workflow-execution/llms.txt).

4. Click **Next**.

### Configure connection details

1. Enter a **Connection Name** to identify your CrateDB environment. For example, `production-cratedb`, `analytics-db`, `data-warehouse`.

2. Assign **Connection Admins** to manage access. At least one admin is required.

### Configure crawler settings

Before running the CrateDB crawler, you can configure additional settings:

* **Exclude Metadata**: Select assets you want to exclude from crawling
* **Include Metadata**: Select assets you want to include in crawling
* **Exclude regex for tables & views**: Specify a regular expression to ignore tables and views based on naming conventions
* **Advanced Config**: 
 - **Enable Source Level Filtering**: Enable schema-level filtering at source
 - **Use JDBC Internal Methods**: Enable JDBC internal methods for data extraction

:::info **Did you know?** 
If an asset appears in both the include and exclude filters, the exclude filter takes precedence.
:::

### Run crawler

You can now start extracting metadata from your CrateDB database:

- **Run now**: Click **Run** to start a one-time crawl.
- **Schedule runs**: Click **Schedule Run** to automate recurring crawls (hourly, daily, weekly, or monthly).

Monitor crawl progress in the activity log. Once complete, your CrateDB assets appear in Atlan.

## Troubleshooting

If you encounter connection or authentication issues, see [Connection issues](https://docs.atlan.com/llms/connectors/cratedb/connection-issues/llms.txt).

## See also

* [What does Atlan crawl from CrateDB?](https://docs.atlan.com/llms/connectors/cratedb/what-does-atlan-crawl-from-cratedb/llms.txt)
* [Preflight checks for CrateDB](https://docs.atlan.com/llms/connectors/cratedb/preflight-checks-for-cratedb/llms.txt)

---