310 docs tagged with "data"

Find and view archived assets in Atlan using the filter menu. Archived assets are soft-deleted and removed from search results by default, but can be recovered and accessed through discovery filters.

Add contract impact analysis to GitHub pull requests using the Atlan GitHub Action. Setup requires an API token or OAuth client with persona and metadata policy permissions.

Add custom metadata fields to assets to store organization-specific information. Setup requires admin users to create custom metadata structures first, then users with edit permissions can populate values.

Add descriptions to assets, columns, and tables in Atlan to provide contextual information. Descriptions support Markdown syntax and can be crawled from sources, viewed in asset sidebars, and searched using property filters.

Learn about add impact analysis in gitlab.

:::warning Who can do this? You must be an admin user in Atlan to create options for custom metadata properties.

Guide to Atlan's AI capabilities and automation features for enhanced data governance and productivity.

Security and compliance information for Atlan AI, including AI architecture, data handling, encryption, model management, and compliance frameworks.

Attach tags to assets in Atlan to identify key characteristics and group assets for usage or data protection. Supports tags from imported sources and native tags.

Create profiling playbooks to scan data assets at scale, identify data quality issues, and improve asset quality. Setup requires admin user permissions in Atlan.

Atlan supports column-level lineage generation for cloud data warehouses like Snowflake, Amazon Redshift, Google BigQuery, and more. Atlan [mines SQL queries](/.

You can [create webhooks](/product/integrations/automation/webhooks/how-tos/create-webhooks) in Atlan to configure alerts or triggers for downstream actions for metadata change events, including schema changes. You can also configure alerts for asset creation or deletion events.

A number of Atlan's [supported connectors](/product/connections/references/connectors-and-capabilities) use a JDBC- or REST API-based approach for metadata extraction. If you are attempting to connect to a source with no native integration, [contact Atlan support](/support/submit-request) to share more details about your use case.

You can query any data warehouse (DW) or data lake (DL) if the integration is supported via Atlan's [supported sources](/product/connections/references/supported-sources). Once integrated, you can query the underlying data using the [Insights](/product/capabilities/insights/how-tos/query-data) feature.

Atlan recommends that you turn off sample data preview at a connection level. For example, you can configure the [Snowflake crawler](/apps/connectors/data-warehouses/snowflake/how-tos/crawl-snowflake) to prevent users from previewing any Snowflake data.

Configure ADFS authentication for Amazon Redshift connections using ADFS (Active Directory Federation Services) as the identity provider.

Configure Snowflake data metric functions <Badge variant="preview" text="Private Preview" link="/get-started/references/product-release-stages#private-preview" />

This document provides recommended solutions for integrating Atlan instances hosted on Microsoft Azure with the following:.

You can configure and use [Atlan's metadata-extractor tool](/apps/connectors/database/on-premises-databases/how-tos/set-up-on-premises-database-access) to extract metadata from on-premises databases with Kubernetes deployment architecture, as an alternative to using Docker Compose.

Extract metadata from Aiven Kafka to catalog topics, partitions, and consumer groups. Discover and govern your Kafka messaging infrastructure after configuring permissions.

Extract metadata from your AlloyDB for PostgreSQL database and establish a connection between Atlan and your database

Extract metadata from Amazon Athena to catalog databases, tables, and views stored in S3. Discover and govern your SQL query data lake assets after configuring AWS permissions.

Extract metadata from Amazon DynamoDB to catalog tables, items, and attributes. Extract asset information and column-level details after configuring AWS permissions.

Extract metadata from Amazon MSK to catalog topics, partitions, and consumer groups. Discover and govern your Kafka messaging infrastructure after configuring permissions.

Extract metadata from Amazon QuickSight to catalog dashboards, analyses, and datasets. Discover and govern your BI assets after configuring permissions.

Crawl metadata from Amazon Redshift to catalog tables, schemas, views, and columns. Extract lineage and column-level asset details after configuring user access permissions.

Crawl metadata from AWS Glue Data Catalog to catalog jobs, workflows, tables, and data transformations. Extract lineage information after configuring AWS access permissions.

Configure and run the SMUS Catalog crawler to extract assets, metadata, and lineage from SMUS and catalog and enrich them in Atlan.

Configure and run the Atlan BigID workflow to crawl metadata from BigID.

Create a ClickHouse crawler workflow in Atlan to extract metadata from your database.

Extract metadata from your Cloud SQL for PostgreSQL database and establish a connection between Atlan and your database

Learn about crawl confluent kafka.

Configure and run the CrateDB crawler to extract metadata from your database

Crawl classification metadata from Cyera to enrich assets with data sensitivity and privacy information. Extract data sensitivity classifications and security issues.

Crawl metadata from Databricks to catalog tables, schemas, views, and volumes. Extract lineage and column information after configuring access permissions and authentication.

Discover and catalog AI models registered in the Databricks Unity Catalog Model Registry using Atlan's Databricks connector.

Extract metadata from DataStax Enterprise to catalog databases, tables, columns, and keyspaces. Discover and govern your NoSQL data assets after configuring user permissions.

Crawl metadata from dbt Cloud or dbt Core to catalog models, tests, and documentation. Extract transformation lineage and dbt-generated metadata after configuring project access.

Extract metadata from Domo to catalog dashboards, datasets, cards, and pages. Discover and govern your BI assets after configuring permissions.

Crawl metadata from Dremio to catalog physical datasets, virtual datasets, folders, and spaces. Extract lineage and asset information from your data lakehouse.

Enrich Atlan with Fivetran metadata to track data pipeline lineage and transformations. Configure the Fivetran Platform Connector and connect a supported data warehouse destination.

Crawl metadata from Google BigQuery to catalog datasets, tables, views, and columns. Extract lineage and asset details after completing prerequisite setup.

Configure and run the crawler to extract metadata from Google Cloud Knowledge Catalog.

Crawl metadata from Hightouch to catalog models and syncs. Extract reverse ETL lineage between data warehouse sources and operational business tools after configuring API credentials.

Extract metadata from Hive to catalog databases, tables, views, and columns. Extract lineage and asset information after configuring user permissions.

Extract metadata from IBM Cognos Analytics to catalog reports, queries, and dashboards. Discover and govern your BI assets after configuring user permissions.

Crawl metadata from Iceberg to catalog tables, namespaces, and columns. Extract table metadata and lineage from your Iceberg data lakehouse catalogs.

Crawl metadata from Looker to catalog dashboards, looks, explores, and data sources. Extract usage and lineage information after configuring user permissions.

Crawl metadata from Matillion to catalog jobs, tasks, pipelines, and transformations. Extract lineage information after configuring user permissions and connectivity.

Extract metadata from Metabase to catalog dashboards, questions, and databases. Discover and govern your BI assets after configuring user permissions.

Extract metadata from Microsoft Azure Cosmos DB to catalog databases, containers, and documents. Discover and govern your NoSQL data assets after configuring permissions.

Once you have [configured the Microsoft Azure Data Factory permissions](/apps/connectors/etl-tools/microsoft-azure-data-factory/how-tos/set-up-microsoft-.

Extract metadata from Microsoft Azure Event Hubs to catalog topics, partitions, and consumer groups. Discover and govern your messaging infrastructure after configuring permissions.

Extract metadata from Microsoft Azure Synapse Analytics to catalog databases, schemas, tables, and views. Discover and govern your cloud data warehouse assets after configuring permissions.

Crawl metadata from Microsoft Power BI to catalog workbooks, reports, dashboards, and data sources. Extract lineage and usage metrics after configuring service principal authentication.

Crawl metadata from Microsoft SQL Server to catalog databases, schemas, tables, and views. Extract column details and enable data discovery, lineage tracking, and governance after configuring user permissions.

Extract metadata from MicroStrategy to catalog reports, dashboards, and metrics. Discover and govern your BI assets after configuring authentication.

Extract metadata from Mode to catalog reports, charts, and datasets. Discover and govern your BI assets after configuring user permissions.

Once you have [configured the MongoDB permissions](/apps/connectors/database/mongodb/onprem/how-tos/set-up-mongodb-onprem), you can establish a connection between Atlan and MongoDB.

Configure MongoDB Atlas connection and run the crawler to extract metadata from MongoDB into Atlan.

Crawl metadata from Monte Carlo to capture data quality anomalies and monitors. Extract data quality insights and incident tracking after configuring API credentials.

Crawl metadata from MySQL to catalog databases, tables, schemas, and views. Extract column-level information and document your database structure after configuring user permissions.

Extract metadata from on-premises databases to catalog tables, schemas, and views. Discover and govern your database assets after configuring the metadata-extractor tool.

The Docker-based databricks-extractor offline tool has been sunset. For on-premises or network-restricted Databricks environments, use Self-Deployed Runtime, Secure Agent, or direct connectivity via private link.

Extract metadata from on-premises IBM Cognos Analytics to catalog reports, queries, and dashboards. Discover and govern your BI assets after configuring extractor tool.

Extract metadata from on-premises Kafka to catalog topics, partitions, and consumer groups. Discover and govern your Kafka messaging infrastructure after configuring extractor tool.

Extract metadata from on-premises Looker to catalog dashboards, looks, explores, and data sources. Discover and govern your BI assets after configuring extractor tool.

Extract metadata from on-premises Tableau to catalog workbooks, dashboards, and sheets. Discover and govern your BI assets after configuring extractor tool.

Extract metadata from on-premises ThoughtSpot to catalog pinboards, answers, and liveboards. Discover and govern your BI assets after configuring extractor tool.

Extract metadata from PrestoSQL to catalog databases, tables, views, and columns. Extract query lineage and asset information after configuring user permissions.

Extract metadata from Qlik Sense Cloud to catalog applications, sheets, and visualizations. Discover and govern your BI assets after configuring permissions.

Extract metadata from Qlik Sense Enterprise on Windows to catalog applications, sheets, and visualizations. Discover and govern your BI assets after configuring permissions.

Extract metadata from Redash to catalog queries, dashboards, and visualizations. Discover and govern your BI assets after configuring permissions.

Extract metadata from Redpanda Kafka to catalog topics, partitions, and consumer groups. Discover and govern your Kafka messaging infrastructure after configuring permissions.

Extract metadata from Salesforce to catalog objects, fields, and relationships. Discover and govern your CRM data assets after configuring user permissions.

Extract metadata from SAP HANA to catalog databases, schemas, tables, views, and columns. Discover and govern your enterprise data assets after configuring user permissions.

Extract metadata from Sigma to catalog workbooks, pages, and visualizations. Discover and govern your BI assets after configuring permissions.

Extract metadata from Sisense to catalog dashboards, widgets, and datasets. Discover and govern your BI assets after configuring permissions.

Crawl metadata from Snowflake to catalog tables, schemas, views, and other objects. Extract lineage, column-level details, and documentation after configuring user permissions.

Discover, catalog, and build lineage for AI models registered in the Snowflake Model Registry using Atlan's Snowflake connector.

Crawl metadata from Soda to capture data quality checks, measurements, and test results. Extract data quality insights after configuring API credentials.

Crawl metadata from Starburst Enterprise to catalog catalogs, schemas, tables, and data products. Extract domain definitions and lineage information after configuring user permissions.

Crawl metadata from Tableau to catalog workbooks, dashboards, sheets, and data sources. Extract lineage and usage information after configuring user permissions.

Extract metadata from Teradata to catalog databases, tables, views, and columns. Extract query lineage and asset information after configuring user permissions.

Extract metadata from ThoughtSpot to catalog pinboards, answers, and liveboards. Discover and govern your BI assets after configuring permissions.

Crawl metadata from Trino to catalog schemas, tables, views, and columns. Extract distributed query lineage and asset information after configuring user permissions.

Adding an announcement to your data asset helps you call attention to an important feature or notify others about a change coming down the pipeline. Since announcements in Atlan display the time stamp and author information, you can easily identify whether an announcement is still relevant and who to ask for questions.

Complete guide for connecting Atlan to your data sources, managing integrations, and troubleshooting connection issues.

Data models provide a framework to describe how data is structured, organized, and related within a system. It acts as a blueprint for organizations to design their business applications and processes. Data models can be of different types: relational, hierarchical, entity relationship, and network.

Learn how to connect your data pipelines to Atlan. Explore ETL tools, workflow orchestration, and lineage tracking to build a comprehensive view of your data movement.

:::warning Who can do this? You will need to be an admin user in Atlan to configure these options.

Frequently asked questions about Atlan's Discovery capabilities.

Atlan displays upstream as well as downstream lineage for [Tableau dashboards](/apps/connectors/business-intelligence/tableau/references/what-does-atlan-crawl-f.

Once you've [connected Atlan with Microsoft Excel](/product/integrations/collaboration/spreadsheets/how-tos/integrate-atlan-with-microsoft-excel), you can download impacted assets in Microsoft Excel. This can help you assess the downstream impact of any changes made to an upstream asset for [impact analysis](/product/capabilities/lineage/concepts/what-is-lineage#impact-analysis).

Atlan supports [Snowflake OAuth-based authentication](https://docs.snowflake.com/user-guide/oauth-snowflake-overview) for [Snowflake](/apps/connectors/data-ware.

You will need to [create a client application in Okta](https://help.okta.com/en-us/Content/Topics/Apps/Apps_App_Integration_Wizard_OIDC.htm) to use for [configuring the identity provider in AWS](/apps/connectors/data-warehouses/amazon-redshift/how-tos/enable-sso-for-amazon-redshift).

Credentials are used to obtain an access token from Google's authorization servers for authentication in Atlan.

Configure PingFederate as the External OAuth identity provider so Atlan Insights queries on your Snowflake connection authenticate as each end user.

Set up OAuth SSO authentication for Databricks connections in Atlan.

Beyond the default mapped [dbt Cloud](/apps/connectors/etl-tools/dbt/references/what-does-atlan-crawl-from-dbt-cloud) or [dbt Core](/apps/connectors/etl-tools/dbt/references/what-does-atlan-crawl-from-dbt-core) properties, you can update any of Atlan's metadata attributes (except for `name`, `tenantId`, and `qualifiedName`) through your dbt model's `meta` property.

Enrich AWS Glue table statistics in Atlan using S3 Inventory report files. Reference for workflow configuration, S3 inventory format, and statistics-calculation behavior.

Build upstream lineage between Databricks AI models and the datasets, tables, and functions they depend on.

Retrieve lineage from Unity Catalog and usage and popularity metrics from query history or system tables using REST API, offline, or system table extraction methods.

The Docker-based databricks-extractor offline tool has been sunset. For on-premises or network-restricted Databricks lineage extraction, use Self-Deployed Runtime, Secure Agent, or direct connectivity via private link.

Data teams often lack clarity on which data assets can be considered trustworthy, whether these are frequently used, the freshness of the data itself, or how critical these are for enrichment and governance.

How to configure permissions for future tables created by any user in Cloud SQL for PostgreSQL

Learn how to generate column and measure to page lineage for Microsoft Power BI by using the PowerBI Columns/Measures -> Pages lineage app with PBIR project files stored in cloud object storage.

Learn how to generate column-to-dataset lineage for Microsoft Power BI by using the PowerBI Columns -> Dataset lineage app with report.json files stored in cloud object storage.

Atlan appends the product name Atlan and a unique ID at the end of each query in a comment. This can help you identify queries from Insights in your database access logs.

By default, any user in Atlan can view all [glossaries](/product/capabilities/governance/glossary/concepts/what-is-a-glossary) and nested categories and terms in the _Glossary_ section.

If you're using an Airflow operator supported by OpenLineage, the OpenLineage events will contain input and output details. This means that you do not have to modify your current DAG implementation and Atlan will be able to generate data lineage.

To learn more about OpenLineage, refer to [OpenLineage configuration and facets](/product/connections/references/openlineage-configuration-and-facets).

To integrate Apache Airflow/OpenLineage with Atlan, complete the following steps. To learn more about OpenLineage, refer to [OpenLineage configuration and facets](/product/connections/references/openlineage-configuration-and-facets).

Atlan extracts job-level operational metadata from Apache Flink and generates job lineage through OpenLineage. To learn more about OpenLineage, refer to [OpenLineage configuration and facets](/product/connections/references/openlineage-configuration-and-facets).

Atlan extracts job-level operational metadata from Apache Spark and generates job lineage through OpenLineage. To learn more about OpenLineage, refer to [OpenLineage configuration and facets](/product/connections/references/openlineage-configuration-and-facets).

The Atlan add-in for Microsoft Excel makes it easy to enrich metadata in bulk for your data assets in Atlan. You can use the Atlan add-in for both the web and desktop versions of Microsoft Excel.

You will need to [configure an incoming link](https://confluence.atlassian.com/adminjiraserver/configure-an-incoming-link-1115659067.html) with an external application - in this case, Atlan. This will allow Atlan to access Jira data, which means that Jira will act as the OAuth provider.

If your Atlan admin has [enabled the governance workflows and inbox module](/product/capabilities/governance/stewardship/how-tos/automate-data-governance) in your Atlan workspace, you can create a ServiceNow integration to allow your users to [grant or revoke data access](/product/capabilities/governance/stewardship/how-tos/automate-data-governance) for governed assets in Atlan or any other data source.

Atlan currently supports usage and popularity metrics for the following connectors:

To [export assets to and bulk enrich metadata from](/product/integrations/collaboration/spreadsheets/how-tos/export-assets) a supported spreadsheet tool,.

To request or revoke data access through ServiceNow inside Atlan, you may first need to link your ServiceNow account. This is done automatically for the user that [set up the ServiceNow integration](/product/integrations/project-management/servicenow/how-tos/integrate-servicenow), but not for other users.

:::warning Who can do this? You must be an admin user to manage custom metadata structures, including defining new ones.

You must have a [Unity Catalog-enabled workspace](https://docs.databricks.com/en/data-governance/unity-catalog/get-started.html) and SQL warehouse configured to import Databricks tags in Atlan.

Atlan imports your [Google BigQuery tags](https://docs.getdbt.com/references/resource-configs/tags) and lets you update your Google BigQuery assets with the imported tags. Note that object tagging in Google BigQuery currently requires [Enterprise edition or higher](https://cloud.google.com/bigquery/docs/editions-intro#editions_features).

You can import your Snowflake tags to Atlan through one-way tag sync. The synced Snowflake tags will be matched to corresponding tags in Atlan through case-insensitive name match and your Snowflake assets will be enriched with their synced tags from Snowflake.

Understand all about the new marketplace Teams flow, including the technical nitty-gritties.

The dbt-action is a custom action designed to perform impact analysis on changes to your dbt models in a [GitHub](/apps/connectors/etl-tools/dbt/how-tos/.

Once you have [crawled assets from Amazon Redshift](/apps/connectors/data-warehouses/amazon-redshift/how-tos/crawl-amazon-redshift), you can mine its query history to construct lineage and retrieve [usage and popularity metrics](/product/capabilities/usage-and-popularity/how-tos/interpret-usage-metrics).

Once you have [crawled assets from ClickHouse](/apps/connectors/database/clickhouse/how-tos/crawl-clickhouse), you can mine its query history to construct lineage.

Once you have [crawled assets from Google BigQuery](/apps/connectors/data-warehouses/google-bigquery/how-tos/crawl-google-bigquery), you can mine its query history to construct lineage.

Learn about mine microsoft azure synapse analytics.

Once you have [crawled assets from PostgreSQL](/apps/connectors/database/postgresql/how-tos/crawl-postgresql), you can mine its query history to construct lineage.

Once you have crawled assets from a supported connector, you can mine query history.

Mine query history from Microsoft SQL Server to construct lineage relationships between your database assets.

Once you have [crawled assets from Snowflake](/apps/connectors/data-warehouses/snowflake/how-tos/crawl-snowflake), you can mine its query history to construct lineage.

Once you have [crawled assets from Teradata](/apps/connectors/database/teradata/how-tos/crawl-teradata), you can mine its query history to construct lineage.

The [order of operations](/product/connections/how-tos/order-workflows#order-of-operations) you run in Atlan is important. Follow the specific workflow sequence outlined below when crawling [data tools](/product/connections/references/supported-sources). The right order particularly ensures that lineage is constructed without needing to rerun crawlers.

Verify prerequisites before setting up the AlloyDB for PostgreSQL connector

Before [running the Amazon Redshift crawler](/apps/connectors/data-warehouses/amazon-redshift/how-tos/crawl-amazon-redshift), you can run [preflight chec.

Before crawling Amazon SageMaker Unified Studio, you can run preflight checks to perform the necessary technical validations.

This check tests for the validity of the [host name URL and API key](/apps/connectors/observability/anomalo/how-tos/integrate-anomalo) you provided. If Atlan is unable to connect to your Anomalo instance, this may indicate that your credentials are either incorrect or invalid.

Before running the AWS Glue crawler, you can run preflight checks to validate credentials and IAM permissions.

This check tests for the validity of the [host name URL and API key](/apps/connectors/observability/bigeye/how-tos/ingest-bigeye-data) you provided. If Atlan is unable to connect to your Bigeye instance, this may indicate that your credentials are either incorrect or invalid.

Before running the ClickHouse crawler, you can run preflight checks to perform the necessary technical validations.

Verify prerequisites before setting up the Cloud SQL for PostgreSQL connector

Before [running the Databricks crawler](/apps/connectors/data-warehouses/databricks/how-tos/crawl-databricks), you can run [preflight checks](/product/co.

Preflight checks for DataStax Enterprise

Atlan uses the [DataSet API](https://developer.domo.com/portal/72ae9b3e80374-list-data-sets) to fetch dataset metadata from Domo.

Learn about preflight checks for fivetran.

Each request requires an OAuth 2.0 access token generated via the [service account key](https://cloud.google.com/docs/authentication#service-accounts).

Before [running the Hive crawler](/apps/connectors/database/hive/how-tos/crawl-hive), you can run [preflight checks](/product/connections/concepts/what-a.

Before [running the Metabase crawler](/apps/connectors/business-intelligence/metabase/how-tos/crawl-metabase), you can run [preflight checks](/product/co.

Before [running the Microsoft Azure Data Factory crawler](/apps/connectors/etl-tools/microsoft-azure-data-factory/how-tos/crawl-microsoft-azure-data-fact.

This check is performed for both [basic](/apps/connectors/data-warehouses/microsoft-azure-synapse-analytics/how-tos/set-up-microsoft-azure-synapse-analytics) and [service principal](/apps/connectors/data-warehouses/microsoft-azure-synapse-analytics/how-tos/set-up-microsoft-azure-synapse-analytics) authentication method.

Before [running the Microsoft SQL Server crawler](/apps/connectors/database/microsoft-sql-server/how-tos/crawl-microsoft-sql-server), you can run [prefli.

Before [running the Mode crawler](/apps/connectors/business-intelligence/mode/how-tos/crawl-mode), you can run [preflight checks](/product/connections/co.

Before running the MongoDB Atlas crawler, you can run preflight checks to verify the connected user has the required MongoDB privileges for metadata extraction.

Before [running the MySQL crawler](/apps/connectors/database/mysql/how-tos/crawl-mysql), you can run [preflight checks](/product/connections/concepts/wha.

Before [running the Oracle crawler](/apps/connectors/database/oracle/how-tos/crawl-oracle), you can run [preflight checks](/product/connections/concepts/.

Before [running the PostgreSQL crawler](/apps/connectors/database/postgresql/how-tos/crawl-postgresql), you can run [preflight checks](/product/connectio.

Before [running the PrestoSQL crawler](/apps/connectors/database/prestosql/how-tos/crawl-prestosql), you can run [preflight checks](/product/connections/.

This check tests for access to datasets and other Qlik objects.

Preflight checks for SAP HANA

Before [running the Snowflake crawler](/apps/connectors/data-warehouses/snowflake/how-tos/crawl-snowflake), you can run [preflight checks](/product/conne.

Learn about preflight checks for soda

Before running the Starburst Enterprise crawler, you can run preflight checks to perform the necessary technical validations.

Before [running the Teradata crawler](/apps/connectors/database/teradata/how-tos/crawl-teradata), you can run [preflight checks](/product/connections/con.

Before [running the Trino crawler](/apps/connectors/database/trino/how-tos/crawl-trino), you can run [preflight checks](/product/connections/concepts/wha.

SSL (Secure Sockets Layer) encryption helps establish a secure connection between your data source and Atlan. Atlan currently supports SSL certificates for [crawling Tableau](/apps/connectors/business-intelligence/tableau/how-tos/crawl-tableau) and [crawling Matillion](/apps/connectors/etl-tools/matillion/how-tos/crawl-matillion).

The Secure Agent is designed with multiple security controls to protect metadata, credentials, and communication between systems. This document outlines its security mechanisms across authentication, encryption, container security, network security, and logging and monitoring.

Complete guide to Atlan's security features, compliance certifications, and data protection capabilities.

:::warning Who can do this? You will need your Amazon Athena or AWS administrator involved - you may not have access yourself to complete these steps.

Atlan supports the [S3 extraction method](/apps/connectors/messaging/on-premises-event-buses/how-tos/set-up-on-premises-kafka-access) for fetching metadata from Aiven Kafka. This method uses Atlan's kafka-extractor tool to fetch metadata.

Configure authentication and connection settings for AlloyDB for PostgreSQL connector

Learn about set up amazon dynamodb.

Learn about set up amazon msk.

Learn about set up amazon quicksight.

Create AWS IAM permissions and credentials for Atlan to access and catalog your S3 buckets and objects.

For all details, see [Databricks documentation](https://docs.databricks.com/administration-guide/cloud-configurations/aws/privatelink.html).

Set up AWS PrivateLink between your Snowflake account and your Atlan tenant, including cross-region configurations.

For all details, see [Databricks documentation](https://learn.microsoft.com/en-us/azure/databricks/administration-guide/cloud-configurations/azure/private-link-simplified?source=recommendations#create-the-workspace-and-private-endpoints-in-the-azure-portal-ui).

Atlan supports the API authentication method for fetching metadata from [Anomalo](https://docs.anomalo.com/integrations/atlan-integration). This method uses an API key to fetch metadata.

Learn about set up aws glue.

Configure AWS authentication and IAM credentials to enable Atlan to integrate with AWS DataZone and SageMaker Catalog Unified Studio.

Create a BigID system user and API token for Atlan integration.

Configure a dedicated ClickHouse user with read-only access so Atlan can connect and extract metadata.

Configure authentication and connection settings for Cloud SQL for PostgreSQL connector

Atlan supports the API authentication method for fetching metadata from Confluent Kafka. This method uses an API key and API secret to fetch metadata.

Create a Cyera API token and obtain credentials for Atlan integration.

Atlan supports three authentication methods for fetching metadata from Databricks. You can set up any of the following authentication methods:.

Set up DataStax Enterprise

Configure authentication tokens in dbt Cloud to enable Atlan to fetch and enrich your assets with dbt metadata.

:::warning Who can do this? You'll need your Domo administrator to complete these steps - you may not have access yourself.

Configure Dremio connection and authentication for Atlan integration.

Configure the Fivetran Platform Connector and destination to enable Atlan to extract Fivetran metadata and logs.

You must be a Google BigQuery administrator to run these commands. For more information, see [Google Cloud's Granting, changing, and revoking access to resources](https://cloud.google.com/iam/docs/granting-changing-revoking-access).

Configure Google Cloud Knowledge Catalog connection and authentication by creating a service account with required permissions, or using Workload Identity Federation.

Configure Google Cloud Storage for secure metadata ingestion with Atlan.

Configure Hightouch roles, groups, and API keys to connect with Atlan.

Configure permissions and authentication for Hive to enable metadata extraction in Atlan.

:::warning Who can do this? You must be an IBM Cognos Analytics administrator to complete these steps - you may not have access yourself.

Configure Generic REST Catalog and BigLake Metastore authentication for Iceberg integration in Atlan.

Create Inventory report for Amazon S3 in case of inventory based ingestion through the crawler.

If your Microsoft Azure Cosmos DB deployment includes a mix of vCore- and RU-based accounts, you must configure both to fetch metadata. You can then use the _vCore and RU_ deployment option to [crawl your Microsoft Azure Cosmos DB assets](/apps/connectors/database/microsoft-azure-cosmos-db/how-tos/crawl-microsoft-azure-cosmos-db).

Atlan supports service principal authentication for fetching metadata from Microsoft Azure Data Factory. This method requires a client ID, client secret, and tenant ID to fetch metadata.

Atlan supports the following authentication methods for Microsoft Azure Event Hubs:.

Atlan supports crawling the following with the Microsoft Azure Synapse Analytics package:.

Enable Microsoft Power BI metadata access for Atlan using a service principal and scanner APIs.

Configure authentication and permissions for Microsoft SQL Server to enable Atlan to crawl metadata from your database.

Atlan supports basic authentication and API token authentication methods for fetching metadata from MicroStrategy.

Atlan supports SCRAM authentication (SCRAM-SHA-1 and SCRAM-SHA-256) for fetching metadata from MongoDB. This method uses a [username and password](#create-database-user) to fetch metadata.

Atlan supports the basic authentication method for fetching metadata from MongoDB. This method uses a username and password to fetch metadata

:::warning Who can do this? You will probably need your Monte Carlo [account owner](https://docs.getmontecarlo.com/docs/authorizationmanaged-roles-and-groups).

:::warning Who can do this? You probably need your MySQL administrator to run these commands - you may not have access yourself.

In such cases you may want to decouple the extraction of metadata from its ingestion in Atlan. This approach gives you full control over your resources and metadata transfer to Atlan.

The Docker-based databricks-extractor offline tool has been sunset. For on-premises or network-restricted Databricks environments, use Self-Deployed Runtime, Secure Agent, or direct connectivity via private link.

:::warning Who can do this? You need access to a machine that can run Docker on-premises. You also need your IBM Cognos Analytics instance details, including credentials.

In some cases you won't be able to expose your Kafka instance for Atlan to crawl and ingest metadata. For example, this may happen when security requirements restrict access to sensitive, mission-critical data.

In some cases you won't be able to expose your Looker instance for Atlan to crawl and ingest metadata. For example, this may happen when security requirements restrict access to sensitive, mission-critical data.

In some cases you will not be able to expose your Microsoft Azure Synapse Analytics instance for Atlan to [mine query history from the Query Store](/apps/connectors/data-warehouses/microsoft-azure-synapse-analytics/how-tos/set-up-microsoft-azure-synapse-analytics). For example, this may happen when security requirements restrict access to sensitive, mission-critical data.

In some cases you may not be able to expose your Tableau instance for Atlan to crawl and ingest metadata. For example, this may happen when security requirements restrict access to sensitive, mission-critical data.

In some cases you will not be able to expose your Teradata instance for Atlan to mine query history. For example, this may happen when security requirements restrict access to sensitive, mission-critical data.

In some cases you will not be able to expose your ThoughtSpot instance for Atlan to crawl and ingest metadata. For example, this may happen when security requirements restrict access to sensitive, mission-critical data.

:::warning Who can do this? You will probably need your PostgreSQL administrator to run these commands - you may not have access yourself.

Learn about set up prestosql.

Enable Query Store, export query history, and prepare JSON files in S3 for mining lineage from Microsoft SQL Server.

:::warning Who can do this? You will probably need your Redash administrator to complete the following steps - you may not have access yourself.

Atlan supports the [S3 extraction method](/apps/connectors/messaging/on-premises-event-buses/how-tos/set-up-on-premises-kafka-access) for fetching metadata from Redpanda Kafka. This method uses Atlan's kafka-extractor tool to fetch metadata.

:::warning Who can do this? You will probably need your SAP HANA administrator to run these commands - you may not have access yourself.

Atlan supports the basic authentication method for fetching metadata from Sisense. This method uses a username and password to fetch metadata.

:::warning Who can do this? You need your Snowflake administrator to run these commands - you may not have access yourself. :::.

:::warning Who can do this? You will need your [Soda Cloud administrator](https://docs.soda.io/soda-cloud/roles-and-rights.html) to complete these steps -.

Configure authentication and permissions for Starburst Enterprise to enable Atlan to crawl metadata from your instance.

:::warning Who can do this? You probably need your Tableau administrator to run these commands - you may not have access yourself.

:::warning Who can do this? You need your Teradata administrator to run these commands - you may not have access yourself.

Configure authentication and permissions for Trino to enable Atlan to crawl metadata from your database.

Configure Workload Identity Federation (WIF) to enable Atlan to authenticate with Google BigQuery without using service account keys.

Reference for the parameters and field mappings when PingFederate is the OAuth identity provider for Snowflake query authentication.

To use both IdP- and SP-initiated SSO, add both the URLs provided.

Star your most-used assets to bookmark them for quick access. Starred assets appear in a personalized widget, can be sorted by star count, and can trigger notifications for metadata updates via Slack or Microsoft Teams.

Use Atlan's structured search to find assets with precision using keywords, filters, and facets. Filter by source, certification, owner, tags, and save searches for sharing and quick access.

The metadata-extractor tool supports the following connection types.

Complete guide to managing tags, classifications, and metadata in Atlan for effective data governance and organization.

What are the known limitations of data models in Atlan?

Troubleshoot Jira integration issues with error, cause, and solution guidance.

So you've crawled your source, and mined the queries, but lineage is missing. Why?

Once you've [connected Atlan with Google Sheets](/product/integrations/collaboration/spreadsheets/how-tos/integrate-atlan-with-google-sheets), you can import the column metadata for all your data assets in Atlan and make changes to them directly in Google Sheets.

Once you've [connected Atlan with Microsoft Excel](/product/integrations/collaboration/spreadsheets/how-tos/integrate-atlan-with-microsoft-excel), you can import the column metadata for all your data assets in Atlan and make changes to them directly in Microsoft Excel.

Refine asset searches using filters by source, type, domain, certification status, ownership, and custom metadata. Filters help you discover relevant assets faster and can be bookmarked, shared, or used to export assets in bulk.

Once you have [ingested your ER model assets in Atlan](/product/capabilities/data-models/concepts/what-are-data-models), you can:.

You can also view additional details and run status for each query and use filters to track specific queries. Query logs are persisted throughout the lifecycle of the Atlan instance for your organization.

Note that process entities may not have a counterpart entity in Microsoft Power BI. Consider these to be nodes that you can enrich with metadata to describe the process or relationship between two Microsoft Power BI assets.

Reference guide for Cyera metadata crawled by Atlan.

Atlan crawls and maps the following assets and properties from AlloyDB for PostgreSQL

During an Amazon Athena crawl, Atlan extracts metadata for Databases, Schemas, Tables, Views, and Columns. Reference tables map each Athena property to its Atlan asset.

Atlan crawls and maps the following assets and properties from Amazon DynamoDB. Atlan also currently supports lineage between Amazon DynamoDB as a source to supported data warehouses as destinations, as enriched by Fivetran.

Atlan currently supports lineage for the Amazon QuickSight connector to the following data sources:.

During an Amazon Redshift crawl, Atlan extracts metadata for Databases, Schemas, Tables, Views, and Columns. Reference tables map each Redshift property to its Atlan asset.

Atlan crawls and maps the following assets and properties from Anaplan.

Once you have [integrated Anomalo](/apps/connectors/observability/anomalo/how-tos/integrate-anomalo), Atlan will receive webhook events when checks are executed in Anomalo. These checks will be cataloged in Atlan to create a relationship with existing assets using the association information from the check.

Atlan maps the following assets and properties from Apache Flink/OpenLineage. Asset lineage support depends on the data sources that OpenLineage supports.

Atlan maps the following assets and properties from Apache Spark/OpenLineage. Asset lineage support depends on the data sources that OpenLineage supports.

During an AWS Glue crawl, Atlan extracts metadata for Catalogs, Databases, Schemas, Tables, and Jobs. Reference tables map each Glue property to its Atlan asset.

Reference guide for BigID metadata crawled by Atlan.

During a ClickHouse crawl, Atlan extracts metadata for Databases, Schemas, Tables, Views, and Columns. Reference tables map each ClickHouse property to its Atlan asset.

Atlan crawls and maps the following assets and properties from Cloud SQL for PostgreSQL

During a Databricks crawl, Atlan extracts metadata for Databases, Schemas, Tables, Views, Materialized Views, Volumes, External Locations, and AI Models. Reference tables map each Databricks property to its Atlan asset.

What does Atlan crawl from DataStax Enterprise?

Atlan supports lineage for the following asset types:.

During a Fivetran crawl, Atlan creates lineage between source and destination data assets and extracts metadata for Fivetran Connectors. Reference tables map each Fivetran Connector property to its Atlan asset.

During a Google BigQuery crawl, Atlan extracts metadata for Projects, Datasets, Tables, Views, and Materialized Views. Reference tables map each BigQuery property to its Atlan asset.

Learn what assets and metadata Atlan extracts from Hightouch reverse ETL pipelines and syncs.

During a Hive crawl, Atlan extracts metadata for Databases, Schemas, Tables, Views, Materialized Views, and Columns. Reference tables map each Hive property to its Atlan asset.

Once you have [crawled Microsoft Azure Cosmos DB](/apps/connectors/database/microsoft-azure-cosmos-db/how-tos/crawl-microsoft-azure-cosmos-db), you can [.

Atlan crawls and maps the following assets and properties from Microsoft Azure Data Factory.

During a Microsoft Azure Synapse Analytics crawl, Atlan extracts metadata for Databases, Schemas, Tables, Views, and Routines. Reference tables map each Azure Synapse property to its Atlan asset.

Atlan crawls and maps the following assets and properties from Microsoft SQL Server.

Atlan crawls and maps the following assets and properties from MongoDB (self-managed). Atlan currently doesn't support lineage for MongoDB (self-managed) assets.

During a MongoDB crawl, Atlan extracts metadata for Databases, Collections, and Columns. Reference tables map each MongoDB property to its Atlan asset.

During a MySQL crawl, Atlan extracts metadata for Databases, Schemas, Tables, Views, and Stored Procedures. Reference tables map each MySQL property to its Atlan asset.

Atlan crawls and maps NetSuite record types to tables and record fields to columns.

During an Oracle crawl, Atlan extracts metadata for Databases, Schemas, Tables, Views, and Procedures. Reference tables map each Oracle property to its Atlan asset.

During a PostgreSQL crawl, Atlan extracts metadata for Databases, Schemas, Tables, Views, and Functions. Reference tables map each PostgreSQL property to its Atlan asset.

Atlan crawls and maps the following assets and properties from PrestoSQL.

Atlan crawls and maps the following assets and properties from Qlik Sense Cloud.

What does Atlan crawl from SAP BW/4HANA? <Badge variant="preview" text="Private Preview" link="/get-started/references/product-release-stages#private-preview" />

Assets, properties, and data type mappings that Atlan discovers and catalogs from SAP Datasphere.

What does Atlan crawl from SAP ECC? <Badge variant="preview" text="Public Preview" link="/get-started/references/product-release-stages#public-preview" />

What does Atlan crawl from SAP S/4HANA? <Badge variant="preview" text="Public Preview" link="/get-started/references/product-release-stages#public-preview" />

During a Sisense crawl, Atlan extracts metadata for Dashboards, Widgets, Data Models, Data Model Tables, and Folders. Reference tables map each Sisense property to its Atlan asset.

Atlan crawls and maps the following assets from SageMaker Catalog Unified Studio.

During a Snowflake crawl, Atlan extracts metadata for Databases, Schemas, Tables, Views, and Procedures. Reference tables map each Snowflake property to its Atlan asset.

Atlan crawls datasets and then filters out all the datasets without any checks. It then crawls the checks associated with each of the datasets with checks from Soda. These checks are cataloged in Atlan to create a relationship with existing assets using the association information from the dataset.

Atlan crawls and maps the following assets and properties from Starburst Enterprise.

During a Tableau crawl, Atlan extracts metadata for Projects, Workbooks, Worksheets, Dashboards, Datasources, and Flows. Reference tables map each Tableau property to its Atlan asset.

During a Teradata crawl, Atlan extracts metadata for Schemas, Tables, Views, and Columns. Reference tables map each Teradata property to its Atlan asset.

Atlan crawls and maps the following assets and properties from Trino.

With two of your most important workspaces connected, you can save time and improve the way you track issues for your data.

With two of your most important workspaces connected, you can save time and improve the way you share data assets with your team.

Learn about the features and capabilities of the Slack integration with Atlan.

By default, users can search and discover [glossaries](/product/capabilities/governance/glossary/concepts/what-is-a-glossary) in Atlan, irrespective of their user role. The rationale being that glossaries are meant to be accessible to all users who want to understand business context. You can define a [glossary policy](/product/capabilities/governance/custom-metadata/how-tos/control-access-metadata-data#glossary-policies) to control what users can do with glossary metadata and [create a persona](/product/capabilities/governance/access-control/how-tos/create-a-persona) to curate edit access.

Atlan uses Matillion's metadata API to generate lineage associated with [Matillion connectors](https://www.matillion.com/connectors). This is particularly useful for creating lineage between different tools.

Microsoft Azure Data Factory REST API reference for lineage extracted into Atlan.

Learn about what lineage does atlan extract from microsoft azure synapse analytics?.

This document helps you understand how Atlan generates lineage to upstream SQL sources for your Microsoft Power BI assets using a custom query parser, and the steps you can take while developing reports and dashboards in Microsoft Power BI to create seamless lineage generation.

Atlan personnel do not have access to any customer instance unless specifically provided by the customer. Accordingly, in the event that a customer instance contains personal data and Atlan personnel are provided access to that instance, Atlan may act as a personal data processor. In addition, depending on whether the customer is a data controller or processor, Atlan may act as a data processor or subprocessor, respectively.

When [creating a visual query](/product/capabilities/insights/how-tos/query-data), Atlan recommends that you do not select a database or schema in the editor context. Leaving both blank will allow you to discover more tables to join in the Visual Query Builder.

Atlan supports extracting and displaying description metadata for your [Salesforce objects](/apps/connectors/crm/salesforce/references/what-does-atlan-crawl-from-salesforce).

The home icon on top of any asset on the [lineage graph](/product/capabilities/lineage/how-tos/view-lineage) indicates the current asset in focus. The lineage view will be different based on the asset you're viewing. To view column-level lineage for [supported sources](/product/connections/references/supported-sources), click **view columns** and then select a column to view data flows for that particular asset.

Everything about managing data workflows, understanding lineage generation, and optimizing data processing pipelines in Atlan.