Skip to main content

How Atlan connects to SageMaker

Atlan connects to your Amazon SageMaker service to extract metadata including models, jobs, features, and lineage while maintaining network security and compliance. You can choose between Direct connectivity for SageMaker accessible from the internet or Self-deployed runtime for SageMaker that must remain behind your firewall.

Connect via direct network connection​

Atlan's SageMaker workflow establishes a direct network connection to your AWS SageMaker service from the Atlan SaaS tenant. This approach works when your AWS account and SageMaker service can accept connections from the internet.

  • Atlan's SageMaker workflow connects directly to your AWS SageMaker service from the Atlan SaaS tenant over HTTPS.
  • You provide AWS credentials (IAM user or IAM role) when creating a crawler workflow.
  • Atlan executes AWS SageMaker API calls to discover models, training jobs, endpoints, and features, then catalogs the results in Atlan Cloud.
  • Your AWS account accepts inbound API requests from Atlan's IP addresses, controlled through your AWS security groups and IAM policies.

For details on how direct connectivity works, see Direct connectivity.

Connect via self-deployed runtime​

A runtime service deployed within your network acts as a secure bridge between Atlan Cloud and your AWS SageMaker service. This approach works when your AWS account must remain fully isolated behind your firewall or when you require additional network security controls.

  • A runtime service sits within your network perimeter, deployed on Docker Compose or a Kubernetes cluster.
  • The runtime maintains an outbound HTTPS connection to Atlan Cloud and a local network connection to your AWS SageMaker service.
  • When you create a crawler workflow, Atlan Cloud sends metadata extraction requests to the runtime through the outbound connection.
  • The runtime translates requests into AWS SageMaker API calls, executes them on your SageMaker service, and returns the results to Atlan Cloud.
  • Your AWS account never exposes endpoints to the internet—all connections are initiated from within your network.
  • In Agent mode, you must configure AWS credentials through your secret store since the credentials aren't directly accessible from Atlan Cloud.

For details on how Self-Deployed Runtime works, see SDR connectivity.

How it protects your data​

SageMaker services contain metadata about your machine learning models, training jobs, and features. Atlan's connection architecture protects your environment through multiple security layers.

  • Metadata extraction, not data replication: Atlan extracts only structural metadata—model names, configurations, job definitions, feature definitions, and lineage relationships. The actual model artifacts, training data, and inference data remain in your AWS account. For example, if you have a SageMaker model trained on customer data, Atlan discovers the model metadata and lineage, but never accesses or stores the actual training data or model artifacts.

  • Read-only operations: All AWS SageMaker API calls are read-only. The connector can't create or delete models, modify training jobs, or change any SageMaker configuration. The AWS IAM permissions you grant control exactly what the connector can access.

  • Credential encryption: AWS credentials are encrypted at rest and in transit. In Direct connectivity, Atlan encrypts credentials before storage. In Self-deployed runtime, credentials never leave your network perimeter—the runtime retrieves them from your enterprise-managed secret vaults (AWS Secrets Manager, Azure Key Vault, GCP Secret Manager, or HashiCorp Vault) only when needed, and Atlan Cloud never receives or stores them.

  • Network isolation with Self-deployed runtime: Your AWS SageMaker service gains complete network isolation from the internet. The service only accepts connections from the runtime within your local network. The runtime itself only makes outbound HTTPS connections to Atlan Cloud, which your network team can control through firewall rules.

See also​