Set up Amazon S3

This guide walks you through creating IAM permissions and authentication credentials to allow Atlan to catalog your S3 buckets and objects.

warning

This integration catalogs only S3 buckets and objects. It doesn't support data lineage.

Prerequisites

Before you begin:

• Set up S3 inventory reports, required only if you plan to use inventory-based ingestion.

Permissions required

To complete this setup, you'll need:

• AWS Administrator access to create IAM policies and users/roles in AWS Management Console
• Atlan workflow access to configure connectors and workflows in Atlan
• Access to configure S3 inventory reports only if you plan to use inventory ingestion

Create IAM policy

Choose the appropriate policy depending on your ingestion method.

Direct ingestion

1. In AWS, go to IAM → Policies

2. Click Create policy

3. Select the JSON tab and paste:

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Sid": "AllowListBuckets",
         "Effect": "Allow",
         "Action": ["s3:ListAllMyBuckets"],
         "Resource": ["arn:aws:s3:::*"]
       },
       {
         "Sid": "AllowAccessToBuckets",
         "Effect": "Allow",
         "Action": [
           "s3:GetBucketLocation",
           "s3:ListBucket",
           "s3:GetObject",
           "s3:GetEncryptionConfiguration",
           "s3:GetBucketVersioning"
         ],
         "Resource": [
           "arn:aws:s3:::<s3_bucket>",
           "arn:aws:s3:::<s3_bucket>/*"
         ]
       }
     ]
   }

4. Replace <s3_bucket> with your actual bucket name or pattern.

5. Click Next, name your policy (e.g. AtlanS3CrawlerDirectPolicy), and create it.

Inventory ingestion

1. In AWS, go to IAM → Policies

2. Click Create policy

3. Select the JSON tab and paste:

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Sid": "AllowInventoryAccess",
         "Effect": "Allow",
         "Action": [
           "s3:ListBucket",
           "s3:GetObject",
           "s3:SelectObjectContent"
         ],
         "Resource": [
           "arn:aws:s3:::<s3_bucket>",
           "arn:aws:s3:::<s3_bucket>/*"
         ]
       }
     ]
   }

4. Replace <s3_bucket> with your actual bucket name or pattern.

5. Click Next, name your policy (e.g. AtlanS3CrawlerInventoryPolicy), and create it.

Set up authentication

Choose between IAM user (simpler) and IAM role (more secure and recommended for production).

IAM user

1. In AWS, go to IAM → Users
2. Click Add users, give a name (e.g. atlan-s3-crawler)
3. Select Attach policies directly and choose the policy you just created
4. Complete the steps and create an access key
5. Save the Access Key ID and Secret Access Key — you'll need them in Atlan

IAM role

1. Contact Atlan support for the Node Instance Role ARN of your Atlan EKS cluster

2. In AWS, go to IAM → Roles → Create role

3. Select Trusted entity type: AWS account

4. Enter Atlan’s AWS account ID (available via support)

5. Attach the policy you created earlier

6. Name the role (e.g. AtlanS3CrawlerRole) and create it

7. Edit the trust relationship with this policy:

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Principal": {
           "AWS": "<atlan_nodeinstance_role_arn>"
         },
         "Action": "sts:AssumeRole"
       }
     ]
   }

8. Share the role name and AWS account ID with Atlan support

9. Once Atlan confirms access, copy the Role ARN (e.g. arn:aws:iam::<account-id>:role/<role-name>) for use in the workflow

warning

Wait for confirmation from Atlan before proceeding to workflow configuration.

Need help?

• Check AWS IAM documentation for detailed reference
• Contact Atlan support for help with setup or integration

Next steps

Crawl S3 assets: Configure your workflow and crawl S3 assets.