Skip to main content

How Atlan connects to AWS Glue

Atlan connects to your AWS Glue Data Catalog to extract technical metadata while maintaining network security and compliance. You can choose Direct connectivity when the AWS Glue API is reachable from the internet or Self-deployed runtime when API calls must originate from within your network.

Connect via direct network connection​

Atlan's AWS Glue workflow establishes a direct connection to the AWS Glue API from the Atlan SaaS tenant. This approach works when your AWS account and Glue endpoints are accessible from the internet (for example, via public AWS API endpoints).

  • Atlan's AWS Glue workflow connects to the AWS Glue API over HTTPS (port 443) from the Atlan SaaS tenant.
  • You authenticate using IAM user credentials (access key and secret key) or IAM role delegation (role ARN and optional external ID). Atlan uses these credentials only to call Glue APIs for metadata; no business data is read or stored.

For details on how direct connectivity works, see Direct connectivity.

Connect via self-deployed runtime​

A runtime service deployed within your organization acts as a secure bridge between Atlan Cloud and the AWS Glue API. Use this approach when you need metadata extraction to run from your network (for example, when AWS Glue is in a private VPC or your policy requires all AWS API calls to originate from your infrastructure).

  • The runtime maintains an outbound HTTPS connection to Atlan Cloud (port 443) and uses your environment's network to call the AWS Glue API over HTTPS (port 443).
  • When you run a crawler workflow with Agent extraction, the runtime retrieves AWS credentials from your enterprise secret store (AWS Secrets Manager, HashiCorp Vault, or similar), calls the Glue API from within your network, and returns only metadata to Atlan Cloud. Your credentials never leave your network.

For details on how Self-Deployed Runtime works, see SDR connectivity.

Security​

Atlan extracts only structural metadata from AWS Glue: catalogs, databases, tables, columns, jobs, and workflows. Business data in your Glue tables or job runs is never read or stored in Atlan.

  • Read-only API usage: The connector uses read-only Glue API operations to list and describe resources. It doesn't create, modify, or delete any Glue resources. IAM permissions you grant control exactly what the connector can discover.

  • Credential handling: AWS credentials are encrypted in transit and at rest. With Direct connectivity, Atlan stores encrypted credentials for workflow execution. With Self-deployed runtime, credentials are never sent to Atlan; the runtime fetches them from your secret store when needed.

  • Network isolation with Self-deployed runtime: All calls to the AWS Glue API originate from your network. The runtime only makes outbound HTTPS connections to Atlan Cloud, which you can control via firewall rules.

See also​