## Crawl MongoDB (self-managed)

URL: https://docs.atlan.com/apps/connectors/database/mongodb/onprem/how-tos/crawl-mongodb-onprem

> Once you have [configured the MongoDB permissions](/apps/connectors/database/mongodb/onprem/how-tos/set-up-mongodb-onprem), you can establish a connection between Atlan and MongoDB.

Create a MongoDB (self-managed) crawler workflow to extract and catalog metadata from your MongoDB databases, Collections, and columns in Atlan. This guide walks you through configuring the connection, setting up extraction methods, and running the crawler.

## Prerequisites

Before you begin, make sure you have:
* Reviewed the [order of operations](https://docs.atlan.com/llms/catalog/connector-capabilities/order-workflows/llms.txt) for connecting data sources to Atlan.
* [Set up MongoDB (self-managed)](https://docs.atlan.com/llms/connectors/mongodb-onprem/set-up-mongodb-onprem/llms.txt) and created a database user with appropriate permissions.

## Create crawler workflow

To create a MongoDB crawler workflow:

1. In your Atlan workspace, click **Connectors** in the left sidebar.
 * If you are using the **Old UI (Classic)**, click **New Workflow** in the top navigation.

2. Click **Marketplace**.

3. Search for **MongoDB Assets** and select it.

4. Click **Install**.

5. Once installation completes, click **Setup Workflow** on the same tile.

### Choose extraction method

Choose your extraction method and provide the connection details.

### Direct

In **Direct** extraction, Atlan connects to your database and crawls metadata directly.

1. For _MongoDB host name_, enter the hostname or IP address of your MongoDB server. This is the network address where your MongoDB instance is running. For replica sets or sharded clusters, you can specify multiple hosts separated by commas. Learn more about [MongoDB connection strings](https://www.mongodb.com/docs/manual/reference/connection-string/#std-label-connections-standard-connection-string-format).

2. For _Port_, enter the port number on which MongoDB is listening. The default port is `27017`.

3. For _Username_, enter the username of the database user you created for Atlan.

4. For _Password_, enter the password for the database user specified in the _Username_ field. The password is used for SCRAM authentication to verify your identity when connecting to MongoDB.

5. For _Authentication database_, enter the name of the database where the user credentials are stored. Typically, this is `admin`, but it can be any database where the user was created. Learn more about [authentication databases in MongoDB](https://www.mongodb.com/docs/manual/core/security-users/#authentication-database).

6. For _Authentication Mechanism_, select the SCRAM authentication method your MongoDB server supports:
 - **SCRAM-SHA-256** (recommended): Uses SHA-256 hashing algorithm for password verification. This is the default authentication mechanism for MongoDB 4.0 and later.
 - **SCRAM-SHA-1**: Uses SHA-1 hashing algorithm. Supported for backward compatibility with older MongoDB versions.

 The authentication mechanism must match what your MongoDB server is configured to use. Learn more about [SCRAM authentication in MongoDB](https://www.mongodb.com/docs/manual/core/security-scram/).

7. For _SSL_, select whether to use SSL/TLS encryption for the connection:
 - **Yes**: Enables SSL/TLS encryption for secure communication between Atlan and your MongoDB server. Use this when your MongoDB instance requires encrypted connections.
 - **No**: Disables SSL/TLS encryption. Use this only if your MongoDB instance doesn't require encrypted connections.

8. For _CA certificate_, if SSL is enabled, provide the Certificate Authority (CA) certificate file that was used to sign your MongoDB server's certificate. The raw content of the file needs to be copied (*-----BEGIN CERTIFICATE-----xxxxxxx-----END CERTIFICATE-----*)

9. For _Certificate key file_, if SSL is enabled and your MongoDB server requires **client authentication**, provide the contents to the client certificate key file. This is used for mutual TLS (mTLS) authentication where both the client and server present certificates. This field is optional and only needed if your MongoDB server is configured to require client certificates.

10. Click the **Test Authentication** button to confirm connectivity to MongoDB.

11. Once authentication is successful, navigate to the bottom of the screen and click **Next**.

### Agent

In **Agent** extraction, Self-Deployed Runtime executes metadata extraction within your organization's environment.

1. Install Self-Deployed Runtime if you haven't already:
 - [Install via Docker Compose](https://docs.atlan.com/llms/platform/self-deployed-runtime/install-on-docker-compose/llms.txt)
 - [Install on Kubernetes](https://docs.atlan.com/llms/platform/self-deployed-runtime/install-on-kubernetes/llms.txt)
2. Select the **Agent** tab.

3. Store sensitive information in the secret store configured with the Self-Deployed Runtime and reference the secrets in the corresponding fields. For more information, see [Retrieve credentials](https://docs.atlan.com/product/capabilities/build-apps/references/retrieve-credentials).

4. For details on individual fields, refer to the Direct extraction tab.

5. Click **Next** after completing the configuration.

### Configure connection

To complete the MongoDB connection configuration:

1. Provide a _Connection Name_ that represents your source environment. For example, you might use values like `production`, `development`, `gold`, or `analytics`.

2. To change the users who are able to manage this connection, change the users or groups listed under _Connection Admins_. If you don't specify any user or group, no one can manage the connection, not even admins.

3. Navigate to the bottom of the screen and click **Next** to proceed.

### Configure crawler

Before running the MongoDB crawler, you can further configure it.

On the _Metadata Filters_ page, you can override the defaults for any of these options. If an asset appears in both the include and exclude filters, the exclude filter takes precedence.

- To select the **databases** you want to include in crawling, click **Include Metadata**. This defaults to all databases if none are specified.
- To select the **databases** you want to exclude from crawling, click **Exclude Metadata**. This defaults to no databases if none are specified.
- To have the crawler ignore Collections based on a naming convention within the included databases, specify a regular expression in the **Exclude regex for collections** field.
 - For example: `_order*|customer*_`
- To set the number of documents to sample from each Collection for field inference, adjust the value in the **Sampling Size** field. For details on how this parameter affects extraction performance and field inference accuracy, see [What does the sampling size workflow setting affect](https://docs.atlan.com/llms/connectors/mongodb-onprem/field-extraction-and-schema-inference/llms.txt) in the FAQ.

### Run crawler

To run the MongoDB crawler, after completing the previous steps:

- To run the crawler once, immediately, at the bottom of the screen, click the **Run** button.
- To schedule the crawler to run hourly, daily, weekly, or monthly, at the bottom of the screen, click the **Schedule & Run** button.

Once the crawler completes running, you can see the assets on Atlan's asset page.

## See also

- [What does Atlan crawl from MongoDB (self-managed)?](https://docs.atlan.com/llms/connectors/mongodb-onprem/what-does-atlan-crawl-from-mongodb-onprem/llms.txt): Learn about the MongoDB assets and metadata that Atlan discovers and catalogs
- [Field extraction and schema inference](https://docs.atlan.com/llms/connectors/mongodb-onprem/field-extraction-and-schema-inference/llms.txt): Find answers to questions about field extraction, schema inference, permissions, and configuration
- [How Atlan connects to MongoDB (self-managed)](https://docs.atlan.com/llms/connectors/mongodb-onprem/how-atlan-connects-to-mongodb-onprem/llms.txt): Connection protocols, ports, and security

---