Skip to main content

Connect with IAM access keys

TL;DR

The fastest path to connect Atlan to AWS Glue - create a read-only IAM user in your AWS account and give Atlan its key pair.

Connect

The fastest path: create a read-only IAM user in your AWS account and give Atlan its key pair. Three phases, each labeled with who performs it.

Prerequisites

Three phases at glance

PhaseWhat happensOwnerTypical time
1. PolicyCreate the read-only IAM policyAWS admin15 minutes
2. User & keysCreate an IAM user, attach the policy, generate keysAWS admin15 minutes
3. Connect & verifyEnter keys + region in Atlan and testAtlan User15 minutes

Phase 1 - Create IAM policy

Owner: AWS admin

Create a policy with exactly the read-only actions Atlan needs (see also Set up AWS Glue); replace <region> and <account_id> with your values.

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"glue:GetTables", "glue:GetDatabases", "glue:GetTable",
"glue:GetDatabase", "glue:SearchTables", "glue:GetTableVersions",
"glue:GetTableVersion", "glue:GetPartition", "glue:GetPartitions",
"glue:GetUserDefinedFunctions", "glue:GetUserDefinedFunction"
],
"Resource": [
"arn:aws:glue:<region>:<account_id>:tableVersion/*/*/*",
"arn:aws:glue:<region>:<account_id>:table/*/*",
"arn:aws:glue:<region>:<account_id>:catalog",
"arn:aws:glue:<region>:<account_id>:database/*"
]
}
]
}
The Resource ARNs are region-scoped

Each resource line names one region. If the policy says us-east-1 but you configure the Atlan connection for us-west-2, every call is denied - or worse, silently returns nothing. Crawling multiple regions needs the resource lines repeated per region (and one Atlan connection per region - see Special cases).

Phase 2 - Create IAM user and keys

Owner: AWS admin

  1. In the AWS console, create an IAM user (suggested name: atlan-glue-crawler - an atlan- prefix makes audits easier).
  2. Attach the Phase 1 policy. No console access, no other permissions.
  3. Create an access key for the user and copy both values immediately - the secret is displayed only once. Store them in your password manager, then share them with the person doing Phase 3 through a secure channel (not email or chat).
Key rotation is the #1 cause of sudden Glue failures

The recurring script: the connection works for months, then every run fails with UnrecognizedClientException: The security token included in the request is invalid. The cause is almost always keys that were rotated, expired, or deactivated by a security policy - without updating Atlan. Add "update Atlan Glue connection" to your key-rotation runbook now. Better: use the cross-account role and never rotate anything.

Phase 3 - Connect and verify

Owner: Atlan User

  1. In Atlan, open New workflow → AWS Glue Assets and choose Direct extraction.
  2. Enter the AWS Access Key, AWS Secret Key, and the Region of your catalog. Leave the Role ARN field empty for this method.
  3. Click Test Authentication.
  4. Configure include/exclude filters and run the crawler (or Schedule & Run for a recurring crawl).

Success checklist

  • Test Authentication passes
  • All three preflight checks pass (authentication, GetDatabases, GetTables)
  • A first crawl completes and the column counts look right - tables with zero columns usually mean Lake Formation is filtering
  • Your key-rotation runbook mentions Atlan

Next steps

Crawl AWS Glue: Configure and run the crawler to extract metadata from AWS Glue