Skip to main content

How Atlan connects to BigID

Atlan connects to your BigID instance to extract privacy metadata including classifications, policy violations, and data source mappings while maintaining network security and compliance. You can choose between Direct connectivity for BigID instances available from the internet or Self-deployed runtime for instances that must remain behind your firewall.

Connect via direct network connection​

Atlan's BigID workflow establishes a direct network connection to your BigID instance from the Atlan SaaS tenant. This approach works when your BigID instance can accept connections from the internet.

  • Atlan's BigID workflow connects directly to your BigID instance from the Atlan SaaS tenant over HTTPS (port 443).
  • You provide connection details (hostname, API token) when creating a crawler workflow.
  • Atlan executes API calls to discover datasources, classifications, and policy violations, then catalogs the results in Atlan Cloud.
  • Your BigID instance accepts inbound network connections from Atlan's IP addresses, controlled through your firewall rules or network security groups.
  • In Direct mode, you can browse and select BigID datasources from a dropdown list populated by querying the BigID metadata endpoint.

Connect via self-deployed runtime​

A runtime service deployed within your network acts as a secure bridge between Atlan Cloud and your BigID instance. This approach works when your BigID instance must remain fully isolated behind your firewall.

  • A runtime service sits within your network perimeter, deployed on Docker Compose or a Kubernetes cluster.
  • The runtime maintains an outbound HTTPS connection to Atlan Cloud (port 443) and a local network connection to your BigID instance (HTTPS).
  • When you create a crawler workflow, Atlan Cloud sends metadata extraction requests to the runtime through the outbound connection.
  • The runtime translates requests into BigID API calls, executes them on your BigID instance, and returns the results to Atlan Cloud.
  • Your BigID instance never exposes ports to the internet—all connections are initiated from within your network.
  • In Agent mode, you must enter exact BigID datasource names manually as comma-separated values since the metadata endpoint isn't available.

How it protects your data​

BigID instances contain sensitive privacy metadata and data source mappings. Atlan's connection architecture protects your environment through multiple security layers.

  • Metadata extraction, not data replication: Atlan extracts only privacy metadata—classifications, policy violations, datasource mappings, and custom attributes. The actual data in your source systems remains in those systems. For example, if BigID has classified a CUSTOMERS table as containing PII, Atlan discovers the classification and policy information, but never accesses or stores the customer records themselves.

  • Read-only operations: All BigID API calls are read-only. The connector can't modify classifications, create or delete policies, or change any BigID configuration. The BigID system user permissions you grant control exactly what the connector can access.

  • Credential encryption: BigID API tokens are encrypted at rest and in transit. In Direct connectivity, Atlan encrypts credentials before storage. In Self-deployed runtime, credentials never leave your network perimeter—the runtime retrieves them from your enterprise-managed secret vaults (AWS Secrets Manager, Azure Key Vault, GCP Secret Manager, or HashiCorp Vault) only when needed, and Atlan Cloud never receives or stores them.

  • Network isolation with Self-deployed runtime: Your BigID instance gains complete network isolation from the internet. The instance only accepts connections from the runtime within your local network. The runtime itself only makes outbound HTTPS connections to Atlan Cloud, which your network team can control through firewall rules.

See also​