## Crawl Microsoft Azure Event Hubs

URL: https://docs.atlan.com/apps/connectors/messaging/microsoft-azure-event-hubs/how-tos/crawl-microsoft-azure-event-hubs

> Extract metadata from Microsoft Azure Event Hubs to catalog topics, partitions, and consumer groups. Discover and govern your messaging infrastructure after configuring permissions.

Extract metadata from your [Microsoft Azure Event Hubs](https://docs.atlan.com/llms/connectors/microsoft-azure-event-hubs/set-up-microsoft-azure-event-hubs/llms.txt) environment into Atlan to catalog topics, partitions, and consumer groups. After configuring the necessary permissions, you can establish a connection and crawl to extract asset-level details.

To crawl metadata from Microsoft Azure Event Hubs,

 review the [order of operations](https://docs.atlan.com/llms/catalog/connector-capabilities/order-workflows/llms.txt) and then complete the following steps.

## Select the source

To select Microsoft Azure Event Hubs as your source:

1. In your Atlan workspace, click **Connectors** in the left sidebar.
 * If you are using the **Old UI (Classic)**, click **New Workflow** in the top navigation.

2. Click **Marketplace**.

3. Search for **Azure Event Hubs Assets** and select it.

4. Click **Install**.

5. Once installation completes, click **Setup Workflow** on the same tile.

## Provide credentials

Choose your authentication method:

- In **SAS Key**, you will need your [event hub namespace and a connection string-primary key](https://docs.atlan.com/llms/connectors/microsoft-azure-event-hubs/set-up-microsoft-azure-event-hubs/llms.txt) for authentication.
- In **Service Principal**, you will need your [event hub namespace](https://docs.atlan.com/llms/connectors/microsoft-azure-event-hubs/set-up-microsoft-azure-event-hubs/llms.txt) and the following:
 - If only fetching metadata from Microsoft Azure Event Hubs, you will need a [client ID, client secret, and tenant ID](https://docs.atlan.com/llms/connectors/microsoft-azure-event-hubs/set-up-microsoft-azure-event-hubs/llms.txt) for authentication.
 - If fetching metadata from both Microsoft Azure Event Hubs and Apache Kafka, you will need a [connection string-primary key](https://docs.atlan.com/llms/connectors/microsoft-azure-event-hubs/set-up-microsoft-azure-event-hubs/llms.txt) and [client ID, client secret, and tenant ID](https://docs.atlan.com/llms/connectors/microsoft-azure-event-hubs/set-up-microsoft-azure-event-hubs/llms.txt) for authentication.

### SAS key

To enter your Microsoft Azure Event Hubs credentials:

1. For _Extraction method_, _Direct_ is the default selection.

2. For _Select which metadata to fetch_, click **From Only Event hubs** to only fetch metadata from Microsoft Azure Event Hubs or click **From Both Kafka and Event hubs** to fetch metadata from both Microsoft Azure Event Hubs and Apache Kafka.

3. For _Bootstrap servers_, enter the [event hub namespace you copied](https://docs.atlan.com/llms/connectors/microsoft-azure-event-hubs/set-up-microsoft-azure-event-hubs/llms.txt) from Microsoft Azure Event Hubs in the following format - `<your event hub namespace>.servicebus.windows.net:9093`.

4. For _Connection string-primary key_, enter the [connection string-primary key you copied](https://docs.atlan.com/llms/connectors/microsoft-azure-event-hubs/set-up-microsoft-azure-event-hubs/llms.txt) from Microsoft Azure Event Hubs.

5. For _Security protocol_, _SSL_ is the default selection for connecting via a Secure Sockets Layer (SSL) channel.

6. Click the **Test Authentication** button to confirm connectivity to Microsoft Azure Event Hubs.

7. Once authentication is successful, navigate to the bottom of the screen and click **Next**.

### Service principal

To enter your Microsoft Azure Event Hubs credentials:

1. For _Extraction method_, _Direct_ is the default selection.

2. For _Bootstrap servers_, enter the [event hub namespace you copied](https://docs.atlan.com/llms/connectors/microsoft-azure-event-hubs/set-up-microsoft-azure-event-hubs/llms.txt) from Microsoft Azure Event Hubs in the following format - `<your event hub namespace>.servicebus.windows.net:9093`.

3. For _Select which metadata to fetch_, you can either:
 - Click **From Only Event hubs** to only fetch metadata from Microsoft Azure Event Hubs. If you choose not to fetch metadata from Apache Kafka, note that Atlan will not be able to display the message count for your event hubs and consumer groups. To enter your credentials: 
 1. For _Client ID_, enter the [application (client) ID you copied](https://docs.atlan.com/llms/connectors/microsoft-azure-event-hubs/set-up-microsoft-azure-event-hubs/llms.txt) for the service principal.
 2. For _Client Secret_, enter the [client secret you copied](https://docs.atlan.com/llms/connectors/microsoft-azure-event-hubs/set-up-microsoft-azure-event-hubs/llms.txt) for the service principal.
 3. For _Tenant ID_, enter the [directory (tenant) ID you copied](https://docs.atlan.com/llms/connectors/microsoft-azure-event-hubs/set-up-microsoft-azure-event-hubs/llms.txt) for the service principal.
 - Click **From Both Kafka and Event hubs** to fetch metadata from both Microsoft Azure Event Hubs and Apache Kafka and then enter your credentials:
 1. For _Connection string-primary key_, enter the [connection string-primary key you copied](https://docs.atlan.com/llms/connectors/microsoft-azure-event-hubs/set-up-microsoft-azure-event-hubs/llms.txt) from Microsoft Azure Event Hubs.
 2. For _Client ID_, enter the [application (client) ID you copied](https://docs.atlan.com/llms/connectors/microsoft-azure-event-hubs/set-up-microsoft-azure-event-hubs/llms.txt) for the service principal.
 3. For _Client Secret_, enter the [client secret you copied](https://docs.atlan.com/llms/connectors/microsoft-azure-event-hubs/set-up-microsoft-azure-event-hubs/llms.txt) for the service principal.
 4. For _Tenant ID_, enter the [directory (tenant) ID you copied](https://docs.atlan.com/llms/connectors/microsoft-azure-event-hubs/set-up-microsoft-azure-event-hubs/llms.txt) for the service principal.
4. For _Security protocol_, _SSL_ is the default selection for connecting via a Secure Sockets Layer (SSL) channel.

5. Click the **Test Authentication** button to confirm connectivity to Microsoft Azure Event Hubs.

6. Once authentication is successful, navigate to the bottom of the screen and click **Next**.

## Configure the connection

To complete the Microsoft Azure Event Hubs connection configuration:

1. Provide a _Connection Name_ that represents your source environment. For example, you might use values like `production`,`development`,`gold`, or `analytics`.

2. (Optional) To change the users who are able to manage this connection, change the users or groups listed under _Connection Admins_.

 :::warning

 If you do not specify any user or group, no one will be able to manage the connection - not even admins.

 :::

3. Navigate to the bottom of the screen and click **Next** to proceed.

## Configure the crawler

Before running the Microsoft Azure Event Hubs crawler, you can further configure it.

On the _Metadata_ page, you can override the defaults for any of these options:

- For _Skip internal event hubs_, keep the default option **Yes** to skip internal event hubs or click **No** to enable crawling them.
- To select the assets you want to exclude from crawling, click **Exclude event hubs regex**. (This will default to no assets, if none specified.)
- To select the assets you want to include in crawling, click **Include event hubs regex**. (This will default to all assets, if none are specified.)

:::info **Did you know?** 
 If an asset appears in both the include and exclude filters, the exclude filter takes precedence.

:::

## Run the crawler

To run the Microsoft Azure Event Hubs crawler, after completing the steps above:

- To run the crawler once, immediately, at the bottom of the screen, click the **Run** button.
- To schedule the crawler to run hourly, daily, weekly, or monthly, at the bottom of the screen, click the **Schedule & Run** button.

Once the crawler has completed running, you will see the assets on Atlan's asset page! 🎉

---