Skip to main content

Special cases

TL;DR

On-premises PostgreSQL, many databases behind one PrivateLink, Aurora failovers and moving IPs, Azure and GCP hosting, cross-region IAM authentication, and VPC Lattice.

Connect

On-premises servers, many databases behind one endpoint, Aurora clusters and their moving IP addresses, Azure and GCP-hosted PostgreSQL, IAM across regions, and VPC Lattice. All of these come from real support history.

On-premises PostgreSQL​

Two workable patterns:

  • Self-deployed agent (recommended). Run Atlan's agent inside your network; it reaches the database locally and pushes results out over HTTPS. No inbound firewall holes at all. Choose Agent extraction in the crawler setup.
  • Private link via your cloud + a forwarder. If your data center is connected to an AWS VPC (VPN/Direct Connect), you can terminate a PrivateLink in that VPC and forward traffic on to the on-prem server. Caveat from support history: when a reverse proxy (e.g. nginx) does that forwarding, PostgreSQL traffic needs a TCP stream block, not an http block - a plain-HTTP proxy config silently fails with connection timeouts:
stream {
  server {
    listen 5432;
    proxy_pass postgres-server.internal.example-corp.com:5432;
  }
}

One NLB + endpoint service can front several RDS instances or databases, one listener port per instance (5432, 5433, 5434...). Each Atlan connection then uses the same VPCE DNS name with its own port. Three rules learned from support history:

  • Tell support every port. Atlan's side has its own security group; a listener port that was never opened there times out even though your side is perfect. Include new ports whenever you add a database later.
  • Give each database a distinct listener port - two environments configured onto the same port is a real support pattern, and only one of them can win.
  • Port translation (listener ≠ database port) is the fragile variant. Connections on translated ports have broken more than once due to connector-side regressions while 5432→5432 connections on the same NLB kept working - and the failures surfaced as SSL/PAM errors, not port errors. It's supported, but test translated-port connections after Atlan upgrades, and mention the translation explicitly in any support ticket.

Aurora clusters, replicas, and the moving-IP problem​

  • Use cluster endpoints (writer/reader hostnames), never instance IPs, wherever a hostname is accepted. Aurora and RDS instance IPs change on failover, reboot, and maintenance.
  • The weak spot is the NLB target group, which targets IPs: after a failover the pinned IP goes stale and crawls time out even though "nothing changed." Fixes, best first: an RDS proxy as the NLB target; automation that refreshes the target group; or at minimum a runbook step after every failover. This exact sequence - reboot → new IP → workflow timeouts for days - recurs in support history.
  • Crawling a reader endpoint is fine (metadata is read-only). If you repoint a connection between writer and reader hosts, retest.

Azure Database for PostgreSQL​

  • The internet path and the agent path work as described. For private connectivity, see Set up Azure private connectivity to PostgreSQL - a support-assisted VNet peering setup.
  • Insist on hostname-based configuration: peering configured against fixed IP addresses breaks every time the database restarts and gets a new IP.
  • Azure Service Principal authentication is not supported for PostgreSQL connections - IAM-style auth is AWS-only (access keys / role ARN). Use Basic auth for Azure-hosted PostgreSQL.

Google Cloud PostgreSQL​

Cloud SQL and AlloyDB have their own connectors with Private Service Connect paths, and self-managed Postgres on GCP uses the agent - see Set up GCP private connectivity to PostgreSQL. If the agent's host needs a web proxy to reach your Atlan tenant, configure the proxy in the agent's compose file - a missing proxy setting is a known cause of "agent not detected" during workflow setup.

Cross-region IAM authentication​

IAM tokens are signed per region. If your Atlan tenant and your RDS database are in different AWS regions, set the AWS Region field in the connection configuration to the database's region. Cross-region token signing has made IAM auth fail with password-style errors while the network path was perfectly healthy - the field exists specifically for this.

VPC Lattice​

Some customers connect via AWS VPC Lattice instead of classic PrivateLink. It has worked, with two caveats:

  • It's a support-assisted setup - not covered by the published docs today. Go through support.
  • The non-obvious failure mode: the Lattice endpoint and the RDS databases must align on availability zones. Mismatched AZs produced connections that never even try to reach the database until the databases moved into the endpoint's AZ. Raise AZ layout in your first support message if you go this route.