Skip to main content

Special cases

TL;DR

Amazon Athena special cases - PrivateLink, cross-account Glue catalogs, cross-region sources, very large catalogs, and multiple Atlan tenants.

Connect

PrivateLink, cross-account catalogs, cross-region sources, very large catalogs, and multiple Atlan tenants.

By default Atlan reaches athena.<region>.amazonaws.com over AWS's public API endpoints (TLS-encrypted). If your organization prohibits that, Atlan can connect through a VPC interface endpoint so traffic stays on AWS's private network. See Set up a private network link to Amazon Athena.

  1. Atlan User - Raise a support ticket asking for PrivateLink to Athena; include tenant URL and Athena region. Support returns the VPC endpoint ID and the correct host value.
  2. Your AWS admin - If you restrict by source, add a policy condition on the endpoint ID.
  3. Atlan User - Enter the host value support gave you and test.
What actually breaks on PrivateLink

PrivateLink failures are almost never authentication: STS succeeds, then the JDBC connection to port 443 times out, because a security group or subnet on the endpoint (either side) doesn't admit the crawler's traffic. If your test-auth passes but crawls time out, jump straight to JDBC timeouts - and ask Atlan support to verify the endpoint's security group covers your tenant's subnets before you touch your own network.

Cross-account Glue catalogs​

Athena can query a Glue catalog that lives in a different AWS account (it shows up under a catalog name other than AwsDataCatalog). Two things to know:

  • The IAM identity Atlan uses needs Glue/Lake Formation access in the account that owns the catalog, not just the account you connect to.
  • Lineage from BI tools (e.g. QuickSight) resolves against what the Athena crawler cataloged. If your dashboards reference the cross-account catalog name but the crawler only crawled AwsDataCatalog, lineage comes up empty. Crawl the same catalog your BI tool queries.

Cross-region and cross-cloud​

Your Athena region does not need to match your Atlan tenant's region - and an Azure-hosted Atlan tenant can crawl Athena on AWS. The setup steps are identical; state both regions/clouds in your support ticket so any PrivateLink or allowlist work lands in the right place. If you use access keys on an Azure-hosted tenant and see unexpected sts:AssumeRole errors, see the note in Connect with IAM access keys.

Multiple Atlan tenants (dev / prod)​

  • Each tenant has its own node instance role - your trust policy must list every tenant's ARN (or use one role per tenant).
  • The Atlan-side allowlist is per tenant: request it for each one explicitly. Existing connections keep working when you add a tenant; the new statements are appended, not replaced.

Very large catalogs​

Catalogs with hundreds of schemas / tens of thousands of tables hit limits that small ones never see:

  • AWS throttling - Rate exceeded from the Athena APIs. Ask AWS for a quota increase on Athena API calls, and narrow the crawler's include filters.
  • One-hour token expiry - role-delegation credentials last one hour by default; a single very long extraction can outlive them (ExpiredTokenException). Raise the role's MaxSessionDuration and mention long crawls in your support ticket.
  • One corrupt table can poison a run - e.g. a view whose Glue entry is broken (Table StorageDescriptor is null) or an Iceberg table missing its metadata pointer. Exclude the offending schema via filters, fix or drop the corrupt entry, and re-run.
  • Use include filters aggressively - crawl what you govern, not the whole account.

External Hive metastores​

If Athena federates to a non-Glue catalog through a Lambda connector, add lambda:InvokeFunction and lambda:GetFunction for that function to the Atlan policy (see Set up Amazon Athena). Without it, only AwsDataCatalog is visible.