## What does Atlan crawl from Amazon MSK?

URL: https://docs.atlan.com/apps/connectors/messaging/amazon-msk/references/what-does-atlan-crawl-from-amazon-msk

> During an Amazon MSK crawl, Atlan extracts metadata for Topics and Consumer Groups. Reference tables map each Amazon MSK property to its Atlan asset.

Once you've [crawled Amazon MSK](https://docs.atlan.com/llms/connectors/amazon-msk/crawl-amazon-msk/llms.txt), you can [use connector-specific filters](https://docs.atlan.com/llms/catalog/discovery/use-the-filters-menu/llms.txt) for quick asset discovery. The following filters are currently supported for these assets:

- [Topics](https://docs.atlan.com/llms/connectors/apache-kafka/what-does-atlan-crawl-from-apache-kafka/llms.txt) - Message count, size (MB), partition count, and cleanup policy filters
- [Consumer groups](https://docs.atlan.com/llms/connectors/apache-kafka/what-does-atlan-crawl-from-apache-kafka/llms.txt) - Member count and topic name filters

Atlan extracts metadata for Topics and Consumer Groups from Amazon MSK and maps each property to its internal asset model. The tables below list which Amazon MSK properties are read and how they appear on the corresponding Atlan asset. Atlan currently only supports asset-level lineage between topics and consumer groups. Upstream, downstream, and column-level lineage are currently not supported.

## Topics

Atlan maps topics from Amazon MSK to its `KafkaTopic` asset type.

| Source property | Atlan property |
| --- | --- |
| Topic | name |
| PartitionCount | kafkaTopicPartitionsCount |
| ReplicationFactor | kafkaTopicReplicationFactor |
| segment.byte | kafkaTopicSegmentBytes |
| compression.type | kafkaTopicCompressionType |
| cleanup.policy | kafkaLogTopicCleanupPolicy |
| isInternal | kafkaTopicIsInternal |
| sizeInBytes | kafkaTopicSizeInBytes |
| recordCount | kafkaTopicRecordCount |
| retention.ms | kafkaTopicRetentionTimeInMs |

## Consumer groups

Atlan maps consumer groups from Amazon MSK to its `KafkaConsumerGroup` asset type.

:::info **Did you know?** 
 Consumer groups are most likely to show up only in streaming scenarios. This is because if a topic is not being consumed actively, Amazon MSK will purge the consumer group. So, if a consumer group is inactive while the workflow runs in Atlan, it will not be cataloged as an asset.

:::

| Source property | Atlan property |
| --- | --- |
| GROUP | name |
| memberCount | kafkaConsumerGroupMemberCount |
| ReplicationFactor | kafkaTopicReplicationFactor |
| topic\_names | kafkaTopicNames |
| TOPIC | kafkaConsumerGroupTopicConsumptionProperties.topicName |
| PARTITION | kafkaConsumerGroupTopicConsumptionProperties.topicPartition |
| LAG | kafkaConsumerGroupTopicConsumptionProperties.topicLag |
| CURRENT-OFFSET | kafkaConsumerGroupTopicConsumptionProperties.topicCurrentOffset |

---