Skip to main content

How Atlan connects to Starburst Enterprise

Atlan connects to your Starburst Enterprise instance to extract technical metadata while maintaining network security and compliance. You can choose between Direct connectivity for instances available from the internet or Self-deployed runtime for instances that must remain behind your firewall.

Connect via direct network connection​

Atlan's Starburst Enterprise workflow establishes a direct network connection to your instance from the Atlan SaaS tenant. This approach works when your Starburst Enterprise coordinator can accept connections from the internet.

  • Atlan's Starburst Enterprise workflow connects directly to your coordinator from the Atlan SaaS tenant over HTTPS (port 443 by default).
  • You provide connection details (hostname, port, credentials) when creating a crawler workflow.
  • Your Starburst Enterprise coordinator accepts inbound network connections from Atlan's IP addresses, controlled through your firewall rules or network security groups.

For details on how direct connectivity works, see Direct connectivity.

Connect via self-deployed runtime​

A runtime service deployed within your network acts as a secure bridge between Atlan Cloud and your Starburst Enterprise instance. This approach works when your Starburst Enterprise coordinator must remain fully isolated behind your firewall.

  • The runtime maintains an outbound HTTPS connection to Atlan Cloud (port 443) and a local network connection to your Starburst Enterprise coordinator (port 443).
  • The runtime executes SQL queries and data products API calls on your instance and returns the metadata to Atlan Cloud.

For details on how Self-Deployed Runtime works, see SDR connectivity.

Security​

Atlan extracts only structural metadata—catalogs, schemas, tables, views, columns, domains, and data products. For example, if you have a customers table with customer records, Atlan discovers the table structure and column definitions, but never queries or stores the customer records themselves.

  • Read-only operations: All SQL queries are read-only SELECT statements on INFORMATION_SCHEMA and system.metadata tables. The connector can't modify data, create or drop database objects, or change any configuration. The Starburst Enterprise permissions you grant control exactly what the connector can access.

  • Credential encryption: Starburst Enterprise connection credentials are encrypted at rest and in transit. In Direct connectivity, Atlan encrypts credentials before storage. In Self-deployed runtime, credentials never leave your network perimeter—the runtime retrieves them from your enterprise-managed secret vaults (AWS Secrets Manager, Azure Key Vault, GCP Secret Manager, or HashiCorp Vault) only when needed, and Atlan Cloud never receives or stores them.

  • Network isolation with Self-deployed runtime: Your Starburst Enterprise instance gains complete network isolation from the internet. The coordinator only accepts connections from the runtime within your local network. The runtime itself only makes outbound HTTPS connections to Atlan Cloud, which your network team can control through firewall rules.

Summary of required permissions​

What Atlan accessesRequired privilegePurpose
{catalog}.information_schema.schemataSHOW on tables in the catalog (BIAC); no grant needed (file-based)Discover schemas
{catalog}.information_schema.tablesSHOW on tables in the catalog (BIAC); no grant needed (file-based)Discover tables, views
{catalog}.information_schema.columnsSHOW on tables in the catalog (BIAC); no grant needed (file-based)Discover columns
{catalog}.information_schema.viewsSHOW on tables in the catalog (BIAC); no grant needed (file-based)Extract view definitions for lineage
system.metadata.catalogsSELECT on system.metadata.catalogs (BIAC); read-only on system catalog (file-based)Discover available catalogs
system.metadata.materialized_viewsSELECT on system.metadata.materialized_views (BIAC); read-only on system catalog (file-based)Detect materialized views
Data products API (/api/v1/dataProduct/domains)SHOW on domains and data products (BIAC); no grant needed if BIAC not enabledDiscover domains
Data products API (/api/v1/dataProduct/products)SHOW on domains and data products (BIAC); no grant needed if BIAC not enabledDiscover data products, datasets, and dataset columns

For details on configuring these permissions, see Set up Starburst Enterprise.

See also​