Skip to main content

SDR orchestrator Private Preview

Important!

Some capabilities shown here may require additional enablement or licensing. Contact your Atlan representative for details.

The SDR (Self-Deployed Runtime) Orchestrator is a container that runs on your Linux VM and manages the full lifecycle of Atlan connector apps—install, upgrade, and automatic update. You run one container; it manages all the others. Once it's running, every additional connector app is installed and managed from the Atlan UI, with no per-app terminal work.

Why SDR orchestrator​

Without the SDR Orchestrator, deploying each connector app means repeating the same manual steps for every app: generate OAuth credentials, download deployment artifacts, fill in configuration, run docker compose up. For one app, that's manageable. For many, it becomes a maintenance burden—and every upgrade repeats the process.

The SDR Orchestrator solves this by automating steps 2–6 of that process. You configure it once, provide your infrastructure details in a single config.yaml, and every app after that's deployed and upgraded automatically—credential provisioning, container lifecycle management, and health verification included. See What you do vs. what SDR handles for a detailed breakdown.

For single-app evaluation or environments that require full manual control, see Docker Compose or Kubernetes.

Architecture​

The SDR Orchestrator runs as a single container on your Linux VM with two communication channels: outbound to Atlan Cloud, and local to your container engine.

SDR Orchestrator—System Context

Figure 1: SDR Orchestrator system context—the orchestrator container communicates with Atlan Cloud over outbound gRPC and HTTPS, and manages sibling app containers via a local Unix socket.

For Atlan Cloud communication, the orchestrator maintains an outbound gRPC+TLS connection (port 443) to Atlan's Temporal service to receive commands, and outbound HTTPS (port 443) for API calls. No inbound ports are opened—no traffic ever flows into your network from Atlan.

For container management, the orchestrator connects to Docker or Podman via a local Unix socket mounted into the orchestrator container. Through this socket, it creates and manages sibling app containers—the connector apps running on the same VM. All of this communication is local; no network is involved.

Key components​

Three components work together to receive commands from Atlan and act on them locally:

📡

Temporal Worker

Outbound gRPC

Polls Atlan's Temporal service over outbound gRPC. Receives deploy, upgrade, and automatic update commands from Atlan and reports results back. This is the only outbound connection the orchestrator maintains to Atlan Cloud.

🔌

Container Runtime Client

Unix socket

Talks to Docker or Podman via the mounted Unix socket. Pulls images, creates and stops containers, runs health checks, and performs blue-green swaps. All communication is local—no network ports involved. See Docker Engine API or Podman system service docs.

📦

Application

Sibling container

The connector app itself—SAP ERP, Monte Carlo, dbt, and others. Runs as a sibling container on the same VM. The orchestrator handles its entire lifecycle: pull, create, health-check, upgrade, and stop.

Container management​

The orchestrator manages app containers by mounting the Docker or Podman Unix socket into the orchestrator container—giving it direct access to the container engine's command API without root access or elevated capabilities.

How the SDR Orchestrator manages containers via Unix socket

Figure 2: Container management via Unix socket—the SDR Orchestrator sends commands to the Docker/Podman engine through a local socket file.

The socket path depends on your container runtime:

Through the socket, the orchestrator sends commands to pull images, start and stop containers, and check health status. No network ports are involved—all communication between the orchestrator and the container engine is local.

App install and upgrade​

When you install or upgrade a connector app from the Atlan UI, the following sequence takes place:

App Deployment Flow

Figure 3: App deployment flow—user triggers install from the Atlan UI, which dispatches a Temporal workflow to the SDR for image pull, container creation, health check, and callback.

  1. User triggers install from the Atlan UI
  2. Marketplace API creates a deployment record and starts a Temporal workflow on queue atlan-sdr-<deployment-name>
  3. SDR picks up the workflow, pulls the image, creates the container, and runs health checks
  4. On success, the SDR notifies Atlan to update the deployment records

If an app has a running workflow at the time of an upgrade, Temporal's durable execution ensures the workflow automatically resumes from where it left off once the updated app container is running—no work is lost.

Automatic update​

The Atlan reconciler continuously monitors the orchestrator's version. When it detects version drift, it triggers an automatic update using a blue-green swap—the new container is fully health-checked before the current one is replaced.

SDR automatic update using blue-green swap

Figure 4: SDR auto updates using blue-green swap—a new container is created and health-checked before the old one is replaced, ensuring zero downtime.

When a newer SDR image is available, the update follows these steps:

  1. Pull the new SDR image
  2. Create a new container (green) with the same mounts and environment variables
  3. Health-check the green container
  4. Swap: rename current container to old, rename green to current
  5. Notify the Atlan registry
  6. Stop the old container (best-effort)

Design principles​

The SDR Orchestrator is designed around a single constraint: it must operate entirely within your security boundary without requiring any changes to it. Every design decision follows from that.

Outbound-only networking means the orchestrator never opens a listening port. All communication is initiated from within your infrastructure—the orchestrator reaches out to Atlan Cloud, Atlan never reaches in. There are no inbound connections to configure, no firewall rules to add, and no VPN tunnels to maintain.

Unprivileged execution means the orchestrator runs as a non-root user with no --privileged flag and no elevated kernel capabilities. It manages containers through the Unix socket without needing host-level access—the socket provides exactly the permissions it needs, no more.

Docker socket access

Docker socket access grants the ability to manage all containers on the host, which is functionally equivalent to docker group membership. For environments requiring stronger isolation, use rootless Podman where the socket is scoped to a single non-root user.

Single entry point means you run one container and it manages everything else. This keeps the operational surface small. There's only one thing to monitor, restart, or update at the infrastructure level; the orchestrator handles the rest.

Automatic updates means the orchestrator can update itself without manual intervention, using the blue-green swap described in Automatic update. You're never stuck on an outdated version, and updates don't interrupt running workloads.

What you do vs. what SDR handles​

With per-app Docker Compose deployment, every step is your responsibility for every app—and every upgrade repeats the cycle. With the SDR Orchestrator, you handle the initial setup once. Everything after that's automated.

Your one-time setup​

Three steps, done once. After this, you never touch the terminal for individual apps.

Prepare your VM
Install Docker or Podman, get a Docker Hub PAT from Atlan, provision object storage and a secret store. See system requirements.
Create config.yaml
Add Atlan credentials, storage, secret store, and registry settings.
Start the orchestrator
Run one podman run or docker run command. The SDR connects to Atlan and waits for instructions.

SDR handles automatically for every app​

From the moment you click Install in the Atlan UI, the SDR takes over—no manual steps, no terminal commands, no per-app configuration needed.

Provision credentials
Uses shared OAuth credentials to authenticate with Atlan—no manual API calls.
Configure app
Downloads artifacts, generates config from the workflow payload, and runs the configurator.
Launch container
Pulls the app image, starts the container, and runs health checks.
Report to Atlan
Reports deployment success and updates the UI. The app is ready to use.

Security​

The SDR Orchestrator inherits the same security model as all Self-Deployed Runtime deployments—outbound-only networking (no inbound ports, no VPN tunnels), non-root execution, and OAuth 2.0 credentials shared across all apps deployed by the SDR. For full details, see Security and Configure network security.

See also​