Troubleshooting AWS Glue connectivity

✦TL;DR

Symptom-first troubleshooting for AWS Glue connectivity - role assumption failures, invalid credentials, access denied, circuit breaker, and missing columns.

||

|Connect

Resolve common AWS Glue connectivity issues when integrating Glue with Atlan. Find the symptom you're seeing in the quick index - each entry includes a check you can run before opening a ticket.

Quick index

You see	Most likely cause	Jump to
Access denied when assuming role arn:aws:iam::… on Test Authentication	Atlan-side allowlist not done, trust policy wrong, or role-name typo	Role assumption
UnrecognizedClientException: The security token included in the request is invalid	Rotated / expired / deactivated access keys	Invalid credentials
AccessDeniedException … not authorized to perform: glue:GetTables …	IAM policy missing an action or resource line - or the database belongs to another account	Access denied
ERROR process: Delete percentage is more than 80.0. Exiting.	Source returned far less than before - permissions, Lake Formation, or real deletions	Circuit breaker
Crawl "succeeds" but tables have 0 columns	Lake Formation column filtering, or schema stripped at source	Missing columns
Preflight passes, crawl still fails	Preflight tests less than the crawl exercises	Preflight passed

"Access denied when assuming role"

Symptom. Test Authentication fails immediately with:

Failed to test AWS Glue connection: Failed to create AWS Glue client:
Failed to create AWS session: Access denied when assuming role
arn:aws:iam::111122223333:role/atlan-glue-read. Please verify the role ARN
and that the base credentials have permission to assume this role.

Why. The two-key door: assumption needs your trust policy and Atlan's allowlist. Check in this order:

1. Has Atlan support confirmed the allowlisting? If you never sent the role name + AWS account ID (Phase 3 of the role guide), or support hasn't confirmed, this is the cause. A correct trust policy alone isn't enough.
2. Is the role ARN in Atlan exactly the allowlisted one? Character-for-character - two near-identical role names differing by a few letters are easy to mix up.
3. Does the trust policy name the right node instance role? Per tenant! Your prod tenant's ARN differs from dev's. Ask your AWS admin to compare the trust policy JSON against the ARN on your support ticket.
4. External ID mismatch? A value in the trust policy but not in Atlan (or vice versa) fails identically. Both blank or both matching.

Security token invalid in request

Symptom. A connection that worked for months starts failing every run with UnrecognizedClientException.

Why. The access keys stored in Atlan are no longer valid - rotated by a security policy, deactivated, or deleted with the IAM user. AWS rejects the request before any Glue permission is checked.

Fix. Ask your AWS admin whether the Atlan IAM user's keys were rotated (IAM console → user → Security credentials → key age/status). Generate fresh keys, update them in the Atlan connection's credential settings, re-run. Prevent recurrence: add Atlan to the rotation runbook - or switch to the cross-account role, which has no keys to rotate.

Access denied for glue:GetTables

Symptom. Preflight or crawl fails with an error like (ARNs illustrative):

{"__type":"AccessDeniedException","Message":"User:
arn:aws:sts::111122223333:assumed-role/atlan-glue-read/rest_api is not
authorized to perform: glue:GetTables on resource:
arn:aws:glue:us-east-1:111122223333:catalog because no identity-based
policy allows the glue:GetTables action"}

Read the error carefully - it tells you the fix:

• The action named is missing from the policy, or the resource named (note the :catalog resource!) is missing from the policy's Resource list. Compare against the full policy - all four resource lines.
• If the resource's account ID isn't the account you configured, the database is shared from another account - see resource links. The owning account must grant access.
• If the policy looks right, check for SCPs or permission boundaries at the AWS organization level, and whether the policy's region matches the connection's Region field.

"Delete percentage is more than 80.0. Exiting."

Symptom. The workflow fails at the process/publish step with the circuit-breaker message. Extraction itself reported no errors.

Why. The crawl returned drastically less than what Atlan already has - so Atlan refused to mass-archive your catalog. The extraction "working" is exactly why this is confusing: the problem is in what AWS returned, not how.

Check yourself, in order:

1. In the AWS Glue console (right account, right region!), do your databases and tables still exist with the expected names?
2. Were the connection's credentials, role, account, or Region field changed around the date failures began?
3. Run the 2-minute CLI check as the Atlan identity - empty results with no error mean permission or Lake Formation filtering.
4. Did your team intentionally clean up / rename databases at source? Then the breaker is correct - tell support the change is intended and they'll help the workflow through it.

Tables crawl fine but columns are missing

Symptom. Databases and tables appear in Atlan, but with zero columns - or a previously complete catalog suddenly loses all its columns and the circuit breaker trips.

Why. Columns arrive inside each table's StorageDescriptor.Columns field. Two things empty it silently:

• Lake Formation column-level security: the Atlan role has table-level grants but not column access. Fix: grant all-column SELECT/DESCRIBE to the Atlan role in Lake Formation (see the grants).
• Schema stripped at source: a Glue ETL job or AWS crawler re-run overwrote table definitions without column info (Iceberg/Delta/Hive setups) - check CloudTrail for UpdateTable events. See Special cases.

In both cases the connector is faithfully reporting what AWS returned - verify with the CLI check before raising a connector bug.

Preflight passes but crawl fails

Symptom. All three preflight checks are green, yet the scheduled crawl fails.

Why. Preflight validates authentication, GetDatabases, and GetTables on one database. The full crawl touches every database, partitions, table versions, and the publish pipeline - a much larger surface. Common causes ranked:

• Permissions valid for some databases but not others (cross-account links, Lake Formation per-database grants).
• The circuit breaker - a data-volume comparison preflight can't see.
• Very large catalogs (thousands of databases) hitting platform-side timeouts - nothing on your side; open a ticket with the workflow run link so Atlan engineering can tune the workflow.

Opening ticket that gets solved fast

Include these six things - they routinely turn a multi-day exchange into one reply:

• Your Atlan tenant URL and a link to the failing workflow run
• Auth method (access keys / cross-account role / agent) and, for roles, the role ARN configured in Atlan
• The exact error text from the workflow logs (screenshot or copy-paste)
• The AWS account ID and region the connection targets
• Output of aws glue get-tables run as the Atlan identity (the 2-minute check)
• Whether it ever worked - and what changed around the time it stopped (key rotation, IAM changes, Glue cleanups, Lake Formation changes)

Other known issues

What are known limitations of AWS Glue connector?

Atlan currently doesn't support the following:

• Parsing MAP type objects for columns and nested columns.
• Tables stored in Amazon S3 Table Buckets display an Iceberg Table subtype label in Atlan. This reflects the table's storage format and is expected behavior. These assets are cataloged as standard Glue Table entities and support pipeline lineage through Open Lineage.

What does sizeBytes property mean in AWS Glue crawls?

In Atlan, the sizeBytes property is derived from the SizeKey parameter returned by AWS Glue. This value represents the total size of all files associated with a table, in bytes. For example, if a Glue table points to an S3 location with three files, each 1 MB in size, sizeBytes shows 3,145,728 bytes (3 × 1,048,576). For more information, see Parameters set on Data Catalog tables by crawler - AWS Glue.

How do I disable Allow Query or Allow Data Preview for AWS Glue connections?

The AWS Glue connection UI doesn't expose Allow Query and Allow Data Preview toggles. To disable these settings, use the pyatlan SDK:

from pyatlan.client.atlan import AtlanClient
from pyatlan.model.assets import Connection
from pyatlan.model.fluent_search import FluentSearch

client = AtlanClient(
    base_url="https://<your-tenant>.atlan.com",
    api_key="<your_api_token>"
)
search = (
    FluentSearch.select()
    .where(FluentSearch.asset_type(Connection))
    .where(Connection.CONNECTOR_NAME.eq("glue"))
    .include_on_results(Connection.ALLOW_QUERY)
    .include_on_results(Connection.IS_SAMPLE_DATA_PREVIEW_ENABLED)
)
criteria = search.to_request()
criteria.attributes = ["name", "qualifiedName"]
results = client.asset.search(criteria)

for connection in results:
    updated = Connection.updater(
        qualified_name=connection.qualified_name,
        name=connection.name
    )
    updated.allow_query = False
    updated.is_sample_data_preview_enabled = False
    client.asset.save(updated)

This script updates all AWS Glue connections in your tenant. To target a specific connection, filter results by connection.qualified_name before calling client.asset.save.