## Crawl MongoDB Atlas

URL: https://docs.atlan.com/apps/connectors/database/mongodb/how-tos/crawl-mongodb

> Configure MongoDB Atlas connection and run the crawler to extract metadata from MongoDB into Atlan.

Create a crawler workflow to automatically discover and catalog your MongoDB Atlas assets, including databases, collections, and schema metadata.

## Prerequisites

Before you begin, make sure you have:

* [Configured MongoDB user permissions](https://docs.atlan.com/llms/connectors/mongodb/set-up-mongodb/llms.txt) with the required metadata read access
* Connection details from your MongoDB deployment: [SQL interface host name](https://docs.atlan.com/llms/connectors/mongodb/set-up-mongodb/llms.txt), [MongoDB native host](https://docs.atlan.com/llms/connectors/mongodb/set-up-mongodb/llms.txt), default database, authentication database, and credentials
* Reviewed the [order of operations](https://docs.atlan.com/llms/catalog/connector-capabilities/order-workflows/llms.txt) for workflow execution

## Create crawler workflow

Create a new MongoDB Atlas crawler workflow in Atlan by selecting the connector package, configuring your extraction method and connection details, and running the crawler to extract metadata.

1. In your Atlan workspace, click **Connectors** in the left sidebar.
 * If you are using the **Old UI (Classic)**, click **New Workflow** in the top navigation.

2. Click **Marketplace**.

3. Search for **MongoDB Atlas Assets** and select it.

4. Click **Install**.

5. Once installation completes, click **Setup Workflow** on the same tile.

### Configure extraction

Select your extraction method and provide the connection details.

### Direct

In **Direct** extraction, Atlan connects to your database and crawls metadata directly.

 - For _SQL interface host name_, enter the [host name of the SQL (or JDBC) endpoint](https://docs.atlan.com/llms/connectors/mongodb/set-up-mongodb/llms.txt) you copied from your MongoDB database.
 - For _Authentication_, _Basic_ is the default method.
 - For _Username_, enter the [username you created](https://docs.atlan.com/llms/connectors/mongodb/set-up-mongodb/llms.txt) in your MongoDB database.
 - For _Password_, enter the [password you created for the username](https://docs.atlan.com/llms/connectors/mongodb/set-up-mongodb/llms.txt).
 - For _MongoDB native host_, enter the [host name of your MongoDB database](https://docs.atlan.com/llms/connectors/mongodb/set-up-mongodb/llms.txt) you copied.
 - For _Default database_, enter the [name of the default database](https://docs.atlan.com/llms/connectors/mongodb/set-up-mongodb/llms.txt) you copied from your MongoDB database.
 - For _Authentication database_, enter the [name of the authentication database](https://docs.atlan.com/llms/connectors/mongodb/set-up-mongodb/llms.txt) you copied. `admin` is the default; see [authentication databases in MongoDB](https://www.mongodb.com/docs/manual/core/security-users/#authentication-database).
 - For _SSL_, keep **Yes** to connect via SSL or click **No**.
 - Click **Test Authentication** to confirm connectivity to MongoDB, then click **Next**.

### Agent

In **Agent** extraction, Self-Deployed Runtime executes metadata extraction within your organization's environment.

1. Install Self-Deployed Runtime if you haven't already:
 - [Install via Docker Compose](https://docs.atlan.com/llms/platform/self-deployed-runtime/install-on-docker-compose/llms.txt)
 - [Install on Kubernetes](https://docs.atlan.com/llms/platform/self-deployed-runtime/install-on-kubernetes/llms.txt)
2. Select the **Agent** tab.

3. Store sensitive information in the secret store configured with the Self-Deployed Runtime and reference the secrets in the corresponding fields.

4. For details on individual fields, refer to the Direct extraction tab.

5. Click **Next** after completing the configuration.

### Configure connection

Set up connection details including a descriptive name and admin access.

1. Provide a _Connection Name_ that represents your source environment. For example, you might use values like `production`, `development`, `gold`, or `analytics`.

2. To change the users able to manage this connection, update the users or groups listed under _Connection Admins_. If you don't specify any user or group, nobody can manage the connection, including admins.

3. Click **Next** at the bottom of the screen.

### Configure crawler

Configure crawler settings to control which assets to include or exclude. If an asset appears in both filters, the exclude filter takes precedence.

On the _Metadata Filters_ page, you can override the defaults. The options are the same for Direct and Agent extraction; when you use **Agent** extraction, filtering and document sampling run on your Self-Deployed Runtime.

### Direct

1. To select the assets you want to include in crawling, click **Include Metadata**. By default, all assets are included if none are specified.

2. To select the assets you want to exclude from crawling, click **Exclude Metadata**. By default, no assets are excluded if none are specified.

3. To have the crawler ignore collections by naming convention, enter a regular expression in the _Exclude regex for collections_ field (for example `.*_TMP|.*_TEMP|TMP.*|TEMP.*`).

### Agent

In **Agent** extraction, **Include Metadata** and **Exclude Metadata** are available as filter options. If you configure via API or app configuration (for example when the workflow runs on the runtime), use these patterns:

- **Include Metadata** (**Include Filter**): A JSON object. Keys are database names or regex patterns; values are arrays of collection name patterns (regex). Use an empty array `[]` for a database to mean all collections. Syntax: `{"^DB1$": ["^COLL1$", "^COLL2$"]}`.
- **Exclude Metadata** (**Exclude Filter**): Same JSON syntax as Include Filter. Only the selected databases and collections are excluded. To exclude collections by naming convention, use regex patterns in the value arrays. Syntax: `{"^DB1$": ["^COLL1$", "^COLL2$"]}`.

### Run crawler

Run preflight checks to validate your configuration, then execute the crawler immediately or schedule it to run on a recurring basis.

1. To verify [permissions and configuration](https://docs.atlan.com/llms/connectors/mongodb/preflight-checks-for-mongodb-atlas/llms.txt) before running, click **Preflight checks**. This option is available for Direct extraction only.

2. Choose your run option:
 - To run the crawler once immediately, click **Run** at the bottom of the screen.
 - To schedule the crawler to run hourly, daily, weekly, or monthly, click **Schedule & Run** at the bottom of the screen.

Once the crawler completes, you can view the assets in Atlan's asset page.

## Need help

If you encounter issues, refer to the [Troubleshooting MongoDB Atlas connectivity](https://docs.atlan.com/llms/connectors/mongodb/troubleshooting-mongodb-connectivity/llms.txt)to resolve common connection errors. You can also contact the Atlan support team by [submitting a support request](https://docs.atlan.com/support/submit-request).

## See also

* [What does Atlan crawl from MongoDB Atlas](https://docs.atlan.com/llms/connectors/mongodb/what-does-atlan-crawl-from-mongodb/llms.txt): Complete reference of assets and metadata discovered during crawling
* [Preflight checks for MongoDB Atlas](https://docs.atlan.com/llms/connectors/mongodb/preflight-checks-for-mongodb-atlas/llms.txt): Validation checks for permissions and configuration before running the crawler

---