πŸ“œ Our Manifesto
🧰 Backup & Disaster Recovery
πŸ‘¨β€πŸ‘©β€πŸ‘§β€πŸ‘¦ Customer Success & Supporty
πŸ‘¨β€πŸ‘©β€πŸ‘§β€πŸ‘¦ Community

Deploy within Existing VPC

How to deploy Atlan in AWS within existing VPC

πŸ“œ Prerequisites

  • Create an AWS sub-account (optional but recommended). This will both help monitor costs and be more secure, as it will not interfere with production or other workloads running in the existing AWS accounts.

  • Create a user with the following IAM permissions:

    ​IAM User Permissions​

πŸ‘€ Note: You can either create a new IAM user or enable an existing user with the above permissions. After stack creation is complete, you can remove these permissions from the IAM user.

  • A VPC with the following configurations and components

    • 1 VPC

    • 4 subnets

      • 2 private subnets β†’ at least 250 IPs each

      • 2 public subnets β†’ 30 IPs each

    • 1 NAT Gateway with an attached Elastic IP

    • 1 internet gateway attached to the VPC

    • 2 route tables

      • Private route table

        • Route to NAT Gateway

        • Both private Subnets associated with it

      • Public route table

        • Route to internet gateway

        • Both public subnets associated with it

πŸ‘€ Note: Make sure that you are using multi-zone subnets.

πŸ› οΈ A step-by-step guide for Atlan cloud deployment on AWS

STEP 1: Create a new stack

Go to the CloudFormation AWS console, and select the option to "Create a New Stack". For the template source, select "Amazon S3 URL".

Here is the Atlan CloudFormation S3 template URL: https://atlan.s3.ap-south-1.amazonaws.com/deploy/marketplace/cloudformation/templates/main-existing-vpc.yaml​

Cloud Formation AWS console

STEP 2: Specify the stack details

Fill in the parameters needed by the CloudFormation template to create the resources:

  • Deployment Method: The deployment method for installing the product.

    • Online (recommended): The stack will be launched with the cluster having internet access. The product will be accessible via the internet without using a VPN.

    • Airgapped: The cluster won't have any kind of internet access and will be only accessible via VPN. Some of the features won't be available, such as:

      • Slack notifications

      • Chat feature

      • Email notifications (you'll need to configure AWS SES separately)

      If going with the airgapped method, follow these steps.

  • License URL: Enter the License URL that was shared by the Atlan team. The deployment will fail without a License URL.

  • VPC Configuration: The IDs of multiple VPC components.

πŸ‘€ Note:

  • VPC ID: The VPC ID where the stack will be deployed. Ensure that the CIDR range of this VPC is different from your other VPC, which might need to be peered with Atlan's VPC. Also, do not overlap the range with any CIDR block assigned to the IP CIDR to be used by the EKS cluster.

  • Public Subnet A ID: The subnet ID of public subnet A. The subnet should be part of the selected VPC, and should have a minimum of 250 IPs.

  • Public Subnet B ID: The subnet ID of public subnet B. The subnet should be part of the selected VPC, and should have a minimum of 250 IPs.

  • Private Subnet A ID: The subnet ID of private subnet A. The subnet should be part of the selected VPC, and should have a minimum of 250 IPs.

  • Private Subnet B ID: The subnet ID of private subnet B. The subnet should be part of the selected VPC, and should have a minimum of 250 IPs.

  • Public Route Table ID: The route table ID of the public route table. The route table should be part of the selected VPC, and have the two public subnets (mentioned above) associated with it with the route to the internet gateway.

  • Private Route Table ID: The route table ID of the private route table. The route table should be part of selected VPC, and have the two private subnets (mentioned above) associated with it with the route to the NAT gateway.

  • NAT Gateway IP: The Elastic IP attached to the NAT Gateway. The NAT Gateway should be part of the selected VPC and have the route defined in private route table (specified above).

Advanced configuration (optional)

  • EKS Configuration: Configuration of the EKS control plane deployed with the stack.

    • Launch EKS in Private Subnet: When this is set to "True", the EKS control plane will deployed in private subnets.

      • The cluster endpoint is only accessible through your VPC. Worker node traffic to the endpoint will stay within your VPC.

      • The product will be only accessible via VPN after VPC peering or transit gateway is set up.

      • The load balancer will be internal and only accessible with VPC or via VPN after VPC peering is done.

    • EKS Cluster IP CIDR: This is the CIDR block to assign Kubernetes service IP addresses. If you don't specify a block, Kubernetes assigns the addresses 172.20.0.0/16 CIDR. We recommend that you specify a block that does not overlap with resources in other networks peered or connected to your VPC. The block must meet the following requirements:

      • Within one of the following private IP address blocks: 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 100.64.0.0/10 or 198.19.0.0/16

      • Does not overlap with any CIDR block assigned to the VPC that you selected.

      • Netmask between /24 and /12.

      • 10.0.0.0/8 and 192.168.0.0/12 won’t work. Make sure the netmask and CIDR block is between the mentioned ranges.

      • Recommended CIDR range: /16

    • AWS IAM User/Role ARN to Access the EKS Cluster through Kubectl: Enter the ARN value of the IAM user or role to get access to the EKS cluster via kubectl. The user/role deploying the stack will have access to the EKS cluster. Use this to provide access to any other user. Refer to this page for more information.

  • Nodes Configuration: Configurations for the nodes being launched. The nodes are divided into a 70-30 ratio of spot and on-demand capacity.

    • EC2 Instance Type: The instance type for Atlan nodes. t3a.2xlarge is recommended for normal workloads. For large workloads, you can increase this to m5a.4xlarge or m5.4xlarge.

    • Launch Spark Nodes: This sets whether to launch Spark nodes. Set this to "True" if you have Azure ADLS. This is set to "False" by default.

    • Additional Userscript: You can add a Bash script that needs to be executed on every node whenever a new node is added. For this, put the public link to the script in the "Link to additional script to run on nodes while bootstrapping" field. However, this is completely optional and can be left blank.

πŸ‘€ Note: While passing the script link, make sure it is public and accessible over the internet. You can use a signed URL with timeout. Once the stack is created, the script will be copied to the S3 bucket launched via CloudFormation, and it will only be fetched by nodes from S3 instead of the internet.

  • Transit Gateway Configuration: These are the configurations to set up the Transit Gateway with the Atlan VPC. Read this documentation for detailed steps.

Sample Parameter List #1
Sample Parameter List #1

STEP 3: Configure the stack options

After entering the parameters above, click on "Next". You can define optional tags as per your IT or Security compliance guidelines, then click the "Next" button.

Tags

STEP 4: Verify all the details

Click on the two checkboxes, and then click on "Create stack".

Accept

STEP 5: Wait 35-40 minutes for stack creation

Output

If you face any issues, recheck the parameters. Otherwise, reach out to the Atlan support team with the CloudFormation error logs.

Your CloudFormation template is now successful πŸŽ‰

STEP 6: Set up the admin account

After deployment, you will need to set up the organization. Just follow these instructures πŸ‘‡

  1. Access the Atlan Product URL, which is displayed as output in the CloudFormation stack.

  2. Fill out the setup page:

Setup
  1. Log in with the email and password entered on the setup page.

Login

STEP 7: Configure DNS and SSL/TLS (optional)

Looking to configure DNS and SSL/TLS? Here are additional steps to set them up:

​How to configure DNS with ACM​

​How to configure DNS with wildcard private certs​

Now you are ready to invite new users to Atlan 😊