Set up a private network link to Amazon Athena
:::warning Who can do this? You will need your Amazon Athena or AWS administrator involved - you may not have access yourself to complete these steps.
You will need your Amazon Athena or AWS administrator involved - you may not have access yourself to complete these steps.
AWS PrivateLink creates a secure, private connection between services running in AWS. This document describes the steps to set this up between Amazon Athena and Atlan.
Request Atlan's details
Before configuring the connection, you will need the following:
- VPC endpoint ID of the Atlan VPC endpoint in the following format -
vpce-0d90d77d1be568544. This will be required to create the IAM policy. - To enter a hostname for crawling Amazon Athena:
- If private DNS hostnames are enabled, enter the default Athena endpoint in the following format -
https://athena.<region>.amazonaws.com- and it will resolve to your VPC endpoint. - If private DNS hostnames are not enabled, enter the primary DNS name of the Atlan VPC endpoint in the following format -
vpce-<hash>-<hash.>vpce-svc-<hash>.<region>.vpce.amazonaws.com- as retrieved from Atlan support.
- If private DNS hostnames are enabled, enter the default Athena endpoint in the following format -
Request it from Atlan support.
"Private DNS enabled" is a setting on the VPC endpoint itself - since the endpoint lives in Atlan's network, ask on the same support ticket: "Is private DNS enabled on our Athena VPC endpoint, and which host value should we enter in the crawler?" If private DNS is enabled, the public-looking hostname (athena.<region>.amazonaws.com) is the correct value - it quietly resolves to the private endpoint. Only paste the long vpce-… DNS name when support tells you private DNS is not enabled. Entering the wrong one is a common source of connection failures.
Create IAM policy
To create an IAM policy with the necessary permissions, follow the steps in the AWS Identity and Access Management User Guide.
-
Create the policy using the following JSON:
{"Version": "2012-10-17","Statement": [{"Sid": "AllowAthenaListDataCatalog","Effect": "Allow","Action": ["athena:ListDataCatalogs"],"Resource": "*","Condition": {"StringEquals": {"aws:SourceVpce": ["<vpce-endpoint-id>"]}}},{"Sid": "AllowAthenaActions","Effect": "Allow","Action": ["athena:StartQueryExecution","athena:GetQueryResults","athena:DeleteNamedQuery","athena:GetNamedQuery","athena:ListQueryExecutions","athena:StopQueryExecution","athena:GetQueryResultsStream","athena:ListNamedQueries","athena:CreateNamedQuery","athena:GetQueryExecution","athena:BatchGetNamedQuery","athena:BatchGetQueryExecution","athena:GetWorkGroup","athena:GetTableMetadata","athena:GetDatabase","athena:GetDataCatalog","athena:ListDatabases","athena:ListTableMetadata"],"Resource": ["arn:aws:athena:us-east-2:666568140392:datacatalog/*","arn:aws:athena:us-east-2:666568140392:workgroup/*"],"Condition": {"StringEquals": {"aws:SourceVpce": ["<vpce-endpoint-id>"]}}},{"Sid": "AllowGlueActionsViaAthena","Effect": "Allow","Action": ["glue:GetDatabase","glue:GetDatabases","glue:CreateDatabase","glue:GetTables","glue:GetTable","glue:SearchTables","glue:GetTableVersions","glue:GetTableVersion","glue:GetPartition","glue:GetPartitions","glue:GetUserDefinedFunctions","glue:GetUserDefinedFunction"],"Resource": ["arn:aws:glue:us-east-2:666568140392:tableVersion/*/*/*","arn:aws:glue:us-east-2:666568140392:catalog","arn:aws:glue:us-east-2:666568140392:table/*/*","arn:aws:glue:us-east-2:666568140392:database/*"],"Condition": {"ForAnyValue:StringEquals": {"aws:CalledVia": ["athena.amazonaws.com"]}}},{"Sid": "AllowS3ActionsOnDataViaAthena","Effect": "Allow","Action": ["s3:GetBucketLocation","s3:ListBucket","s3:GetObject"],"Resource": ["arn:aws:s3:::demo-wide-world-importers","arn:aws:s3:::demo-wide-world-importers/*"],"Condition": {"ForAnyValue:StringEquals": {"aws:CalledVia": ["athena.amazonaws.com"]}}},{"Sid": "AllowS3ActionsOnMetadataViaAthena","Effect": "Allow","Action": ["s3:GetBucketLocation","s3:GetObject","s3:ListBucket","s3:ListBucketMultipartUploads","s3:ListMultipartUploadParts","s3:AbortMultipartUpload","s3:CreateBucket","s3:PutObject"],"Resource": ["arn:aws:s3:::source-curation-athena-metadata","arn:aws:s3:::source-curation-athena-metadata/*"],"Condition": {"ForAnyValue:StringEquals": {"aws:CalledVia": ["athena.amazonaws.com"]}}}]}- Replace
<vpce-endpoint-id>with the VPC endpoint ID received from Atlan support.
- Replace
-
Attach this policy to the IAM user or role used for authentication. For more information, see Choose authentication mechanism or create a new IAM user by following the steps in the Create an IAM user section.
Create an IAM user
Create an AWS IAM user and attach the policy created above to this user.
To create an AWS IAM user:
- Follow the steps in the AWS Identity and Access Management User Guide.
- On the Set permissions page, attach the policy created in the previous step to this user.
- Refer to managing access keys for IAM users to create an access key for the new user.
- Once the user is created, view or download the user's access key ID and secret access key.
This will be your only opportunity to view or download the access keys. You will not have access to them again after leaving the user creation screen.
The connection is now established. You can now use the DNS name of the Atlan VPC endpoint as the hostname to crawl Amazon Athena in Atlan! 🎉
Verify the connection and troubleshoot timeouts
PrivateLink failures are almost never authentication: the STS handshake succeeds, then the JDBC connection to athena.<region>.amazonaws.com:443 times out because a security group or subnet on the VPC endpoint (on either side) doesn't admit the crawler's traffic.
If Test Authentication passes but crawls time out with private IP addresses (10.x.x.x) in the log, don't rotate keys or touch the trust policy - the identity part already worked. Reply on your support ticket quoting the timeout and ask Atlan to verify the endpoint's security group includes all tenant subnets. This is the most common fix and only Atlan can check it.
If the endpoint is in your network, your cloud team can verify:
aws ec2 describe-vpc-endpoints- confirm the endpoint exists, its state is available, and note its security group.- The endpoint security group allows inbound TCP 443 from the crawler's source range.
- The private DNS setting matches the host value entered in the crawler (see the tip above).
For the full symptom-first flow, see Troubleshooting Athena connectivity.