## Crawl Iceberg

URL: https://docs.atlan.com/apps/connectors/data-lakehouses/iceberg/how-tos/crawl-iceberg

> Crawl metadata from Iceberg to catalog tables, namespaces, and columns. Extract table metadata and lineage from your Iceberg data lakehouse catalogs.

Configure and run the crawler to extract metadata from your [Iceberg](https://docs.atlan.com/llms/connectors/iceberg/set-up-iceberg/llms.txt) data lakehouse assets. After completing the prerequisite setup, you can crawl metadata from Iceberg catalogs, namespaces, tables, and columns. Review the [order of operations](https://docs.atlan.com/llms/catalog/connector-capabilities/order-workflows/llms.txt) for metadata enrichment workflows before starting.

## Prerequisites

Before you begin, make sure you have:

- Completed [Set up Iceberg](https://docs.atlan.com/llms/connectors/iceberg/set-up-iceberg/llms.txt)
- Admin access to your Atlan instance
- The connection values for your chosen mode (Generic REST Catalog or BigLake Metastore)

## Create crawler workflow

To crawl metadata from Iceberg, review the [order of operations](https://docs.atlan.com/llms/catalog/connector-capabilities/order-workflows/llms.txt) and then complete the following steps.

1. In your Atlan workspace, click **Connectors** in the left sidebar.
 * If you are using the **Old UI (Classic)**, click **New Workflow** in the top navigation.

2. Click **Marketplace**.

3. Search for **Iceberg Assets** and select it.

4. Click **Install**.

5. Once installation completes, click **Setup Workflow** on the same tile.

### Configure authentication

Choose one authentication mode and then configure either **Direct** extraction or **Agent** extraction for that mode.

### Generic REST Catalog

Use this mode for REST catalogs that support OAuth2 client credentials.

**Direct extraction**

1. **Extraction method**: Select **Direct**.

2. **Authentication method**: Select **Token**.

3. Enter the required values:
 - **REST Catalog URI**: For example, `https://your-catalog.com/api/rest`
 - **Token**: Enter credentials in the format `client-id:client-secret`
 - **Catalog Name**
 - **Warehouse**
 - **Scope** (if required by your catalog)
4. Click **Test Connection**.

5. Once successful, click **Next**.

**Agent extraction**

1. **Extraction method**: Select **Agent**.

2. Provide the same values as Direct extraction through your configured secret store.

3. Complete runtime configuration by following [How to configure Secure Agent for workflow execution](https://docs.atlan.com/llms/platform/secure-agent/configure-secure-agent-for-workflow-execution/llms.txt).

4. Click **Next**.

### BigLake Metastore (GCP)

Use this mode for Iceberg catalogs backed by Google BigLake Metastore.

**Direct extraction**

1. **Extraction method**: Select **Direct**.

2. **Authentication method**: Select **BigLake Metastore (BLM)**.

3. **GCP authentication type**: Select **Service account key**.

4. Enter the required values:
 - **REST Catalog URI**
 - **Project ID**
 - **Location**
 - **Catalog Name**
 - **Warehouse**
 - **Service account JSON key**
5. Click **Test Connection**.

6. Once successful, click **Next**.

**Agent extraction**

1. **Extraction method**: Select **Agent**.

2. Select **BigLake Metastore (BLM)** and **Service account key**.

3. Provide the same values as Direct extraction through your configured secret store.

4. Complete runtime configuration by following [How to configure Secure Agent for workflow execution](https://docs.atlan.com/llms/platform/secure-agent/configure-secure-agent-for-workflow-execution/llms.txt).

5. Click **Next**.

### Workload Identity Federation (WIF)

**Direct extraction**

1. **Extraction method**: Select **Direct**.

2. **Authentication method**: Select **BigLake Metastore (BLM)**.

3. **GCP authentication type**: Select **Workload Identity Federation (WIF)**.

4. Enter the required values:
 - **REST Catalog URI**
 - **Project ID**
 - **Location**
 - **Catalog Name**
 - **Warehouse** (for example, `gs://<your-bucket-name>`)
 - **Service Account Email**
 - **WIF Pool Provider ID**
 - **Atlan OAuth Client ID**
 - **Atlan OAuth Client Secret**
5. Click **Test Connection**.

6. Once successful, click **Next**.

**Agent extraction**

1. **Extraction method**: Select **Agent**.

2. Select **BigLake Metastore (BLM)** and **Workload Identity Federation (WIF)**.

3. Provide the same values as Direct extraction through your configured secret store.

4. Complete runtime configuration by following [How to configure Secure Agent for workflow execution](https://docs.atlan.com/llms/platform/secure-agent/configure-secure-agent-for-workflow-execution/llms.txt).

5. Click **Next**.

### Configure connection

On this page, define how this Iceberg connection is identified and managed in Atlan.

1. Provide a **Connection Name** that represents your source environment (for example, `production`, `development`, or `iceberg-blm`).

2. To control who can manage this connection, configure **Connection Admins**.

3. Click **Next**.

### Configure crawler

Before running the crawler, optionally customize crawl scope on the _Metadata_ page:

- **Exclude Metadata**: Select specific namespaces and tables to skip.
- **Include Metadata**: Select specific namespaces and tables to include.
- **Preflight checks**: Validate connectivity and permissions before execution.

### Run crawler

After configuration, choose how to run:

- Click **Run** to run once immediately.
- Click **Schedule & Run** to run on a schedule.

### Verify crawled assets

After the crawler completes:

1. Navigate to **Workflows** and open the Iceberg workflow run.

2. Review execution details and logs.

3. Confirm status is **Success**.

Then verify crawled assets from Iceberg in Atlan search and asset views.

## See also

- [What does Atlan crawl from Iceberg](https://docs.atlan.com/llms/connectors/iceberg/what-does-atlan-crawl-from-iceberg/llms.txt): Assets and metadata that Atlan ingests.
- [Preflight checks for Iceberg](https://docs.atlan.com/llms/connectors/iceberg/preflight-checks-for-iceberg/llms.txt): Validation checks run before crawling.

---