## Crawl Amazon DocumentDB

URL: https://docs.atlan.com/apps/connectors/database/documentdb/how-tos/crawl-documentdb

> Once you have configured Amazon DocumentDB permissions, you can establish a connection between Atlan and your DocumentDB cluster to extract and catalog metadata.

Create an Amazon DocumentDB crawler workflow to extract and catalog metadata from your DocumentDB databases, collections, and inferred field schemas in Atlan. Amazon DocumentDB is crawled only through [Self-Deployed Runtime](https://docs.atlan.com/llms/platform/self-deployed-runtime/llms.txt) running in the same VPC as your cluster. This guide walks you through installing the runtime, configuring the connection, and running the crawler.

## Prerequisites

Before you begin, make sure you have:
* Reviewed the [order of operations](https://docs.atlan.com/llms/catalog/connector-capabilities/order-workflows/llms.txt) for connecting data sources to Atlan.
* [Set up Amazon DocumentDB](https://docs.atlan.com/llms/connectors/documentdb/set-up-documentdb/llms.txt) and created a crawl user with appropriate permissions.
* The Amazon DocumentDB CA certificate (the global CA bundle `global-bundle.pem`), if your cluster requires TLS. For details, see [How do I configure TLS CA certificates for DocumentDB?](https://docs.atlan.com/llms/connectors/documentdb/field-extraction-and-schema-inference/llms.txt) in the FAQ.

## Create crawler workflow

To create an Amazon DocumentDB crawler workflow:

1. In the top navigation, click **Marketplace**.
2. Search for **AWS DocumentDB Assets** and select it.
3. Click **Install**.
4. Once installation completes, click **Setup Workflow** on the same tile.

If you navigated away before installation completed, go to **New** > **New Workflow** and select **AWS DocumentDB Assets** to proceed.

### Choose extraction method

Amazon DocumentDB is crawled only through Self-Deployed Runtime. Choose the **Self-deployed runtime** tab below to configure the connection.

### Direct

Direct extraction—where Atlan Cloud connects to your cluster over the internet—isn't supported for Amazon DocumentDB. DocumentDB clusters have no public endpoint by design, so you must crawl them through Self-Deployed Runtime instead. Select the **Self-deployed runtime** tab to continue.

For why this is the only supported method, see [Why is Amazon DocumentDB supported only through self-deployed runtime?](https://docs.atlan.com/llms/connectors/documentdb/connectivity-and-deployment/llms.txt) in the FAQ.

### Self-deployed runtime

In **Self-deployed runtime** extraction, the runtime executes metadata extraction within your organization's environment, inside the same VPC as your DocumentDB cluster.

1. Install Self-Deployed Runtime in the same VPC as your DocumentDB cluster, if you haven't already:
 - [Install via Docker Compose](https://docs.atlan.com/llms/platform/self-deployed-runtime/install-on-docker-compose/llms.txt)
 - [Install on Kubernetes](https://docs.atlan.com/llms/platform/self-deployed-runtime/install-on-kubernetes/llms.txt)

2. On the extraction method step, select the **Agent** tab and choose your Self-Deployed Runtime.

3. Store sensitive information—such as the crawl user password or IAM credentials—in the secret store configured with your Self-Deployed Runtime, then reference those secrets in the corresponding fields. For more information, see [Retrieve credentials](https://docs.atlan.com/product/capabilities/build-apps/references/retrieve-credentials).

4. Provide the connection details:

 1. For _Cluster endpoint_, enter the hostname of your DocumentDB cluster endpoint. This is the network address where your DocumentDB cluster accepts connections.

 2. For _Port_, enter the port number on which DocumentDB is listening. The default port is `27017`.

 3. For _Authentication type_, select the method the runtime uses to authenticate with your cluster:
 - **Basic**: Username and password authentication using SCRAM-SHA-1.
 - **IAM**: AWS Identity and Access Management authentication using the MONGODB-AWS mechanism.

 4. Provide the credentials for the selected authentication type:
 - For **Basic** authentication, enter the _Username_ and _Password_ of the crawl user you created for Atlan.
 - For **IAM** authentication, provide the AWS credentials for the IAM user or role mapped to a DocumentDB user.

 5. For _Authentication database_, enter the name of the database where the user credentials are stored. Typically, this is `admin`, but it can be any database where the user was created.

 6. For _TLS CA Certificate_, provide the Amazon DocumentDB CA certificate (the global CA bundle `global-bundle.pem`, available from [https://truststore.pki.rds.amazonaws.com/global/global-bundle.pem](https://truststore.pki.rds.amazonaws.com/global/global-bundle.pem)) using any of the [input forms supported in SDR mode](https://docs.atlan.com/llms/platform/self-deployed-runtime/configure-workflow-execution/llms.txt): upload it through the Atlan UI, reference a base64-encoded key in your secret store, or reference an `objectstore://` path. Provide the complete bundle, not a single extracted certificate. For details, see [How do I configure TLS CA certificates for DocumentDB?](https://docs.atlan.com/llms/connectors/documentdb/field-extraction-and-schema-inference/llms.txt) in the FAQ.

 7. For _Replica set_, enter the name of the cluster's replica set. The default is `rs0`.

 8. Select **Skip TLS hostname verification** if the runtime reaches your cluster through an SSH tunnel or another intermediate hop where the hostname presented in the TLS certificate doesn't match the address the runtime connects to.

 9. Select **Direct node connection** to connect to a single node rather than discovering the replica set topology. This is the MongoDB `directConnection` driver option—use it when you want the runtime to connect to a specific node instead of performing replica-set discovery. It's a per-connection setting within Self-Deployed Runtime and is unrelated to how Atlan reaches your cluster.

5. Navigate to the bottom of the screen and click **Next**.

### Configure connection

To complete the Amazon DocumentDB connection configuration:

1. Provide a _Connection Name_ that represents your source environment. For example, you might use values like `production`, `development`, `gold`, or `analytics`.
2. To change the users who are able to manage this connection, change the users or groups listed under _Connection Admins_. If you don't specify any user or group, no one can manage the connection, not even admins.
3. Navigate to the bottom of the screen and click **Next** to proceed.

### Configure crawler

Before running the Amazon DocumentDB crawler, you can further configure it.

On the _Metadata Filters_ page, you can control which databases and collections Atlan crawls and how schemas are inferred. If an asset appears in both the include and exclude filters, the exclude filter takes precedence.

- To control which **databases and collections** are crawled, provide include and exclude filters. Each filter is a JSON object that maps a database-name regular expression to a list of collection-name regular expressions. An empty list matches all collections in the matched databases.
 - For example, the following include filter crawls all collections in every database whose name starts with `sales_`:
 ```json
 {"^sales_.*$": []}
 ```
- To set the number of documents to sample from each collection for field inference, adjust the value in the **Sample size** field. The default is `100`. For details on how this parameter affects extraction performance and field inference accuracy, see [What does sample size affect?](https://docs.atlan.com/llms/connectors/documentdb/field-extraction-and-schema-inference/llms.txt) in the FAQ.

For details on how database and collection filters are applied, see [How do database and collection filters work?](https://docs.atlan.com/llms/connectors/documentdb/field-extraction-and-schema-inference/llms.txt) in the FAQ.

### Run crawler

To run the Amazon DocumentDB crawler, after completing the previous steps:

- To run the crawler once, immediately, at the bottom of the screen, click the **Run** button.
- To schedule the crawler to run hourly, daily, weekly, or monthly, at the bottom of the screen, click the **Schedule & Run** button.

Once the crawler completes running, you can see the assets on Atlan's asset page.

## See also

- [What does Atlan crawl from Amazon DocumentDB?](https://docs.atlan.com/llms/connectors/documentdb/what-does-atlan-crawl-from-documentdb/llms.txt): Learn about the DocumentDB assets and metadata that Atlan discovers and catalogs.
- [Field extraction and schema inference](https://docs.atlan.com/llms/connectors/documentdb/field-extraction-and-schema-inference/llms.txt): Find answers to questions about field extraction, schema inference, permissions, and configuration.
- [How Atlan connects to Amazon DocumentDB](https://docs.atlan.com/llms/connectors/documentdb/how-atlan-connects-to-documentdb/llms.txt): Connection protocols, ports, and security.

---