## SDR Orchestrator

URL: https://docs.atlan.com/product/connections/self-deployed-runtime/references/orchestrator-architecture

> Architecture overview, components, communication model, and container management for the SDR Orchestrator on Docker and Kubernetes

# SDR Orchestrator

:::warning Important!
Some capabilities shown here may require additional enablement or licensing. Contact your Atlan representative for details.
:::

The SDR (Self-Deployed Runtime) Orchestrator manages the full lifecycle of Atlan connector apps—install, upgrade, and automatic update. You deploy one orchestrator; it manages all your apps. Once it's running, every additional connector app is installed and managed from the Atlan UI, with no per-app terminal work.

The SDR Orchestrator runs on two platforms:

- **Docker/Podman**:a single container on a Linux VM that manages sibling app containers via the local Docker or Podman socket.
- **Kubernetes**:a Helm-deployed pod that manages app Deployments via the Kubernetes API and Helm CLI.

The core architecture is the same on both platforms—a Temporal worker receives commands from Atlan and acts on them locally. The difference is in how apps are deployed and managed at the runtime layer.

## Why SDR Orchestrator

Without the SDR Orchestrator, deploying each connector app means repeating the same manual steps for every app: generate OAuth credentials, download deployment artifacts, fill in configuration, run `docker compose up` or `helm install`. For one app, that's manageable. For many, it becomes a maintenance burden—and every upgrade repeats the process.

The SDR Orchestrator solves this by automating steps 2–6 of that process. You configure it once, provide your infrastructure details in a single configuration file, and every app after that's deployed and upgraded automatically—credential provisioning, lifecycle management, and health verification included. See [What SDR handles automatically for every app](#what-sdr-handles-automatically-for-every-app) for a detailed breakdown.

For single-app evaluation or environments that require full manual control, see [Docker Compose](https://docs.atlan.com/llms/platform/self-deployed-runtime/install-on-docker-compose/llms.txt) or [Kubernetes](https://docs.atlan.com/llms/platform/self-deployed-runtime/install-on-kubernetes/llms.txt).

## What SDR handles automatically for every app

From the moment you click Install in the Atlan UI, the SDR takes over—no manual steps, no terminal commands, no per-app configuration needed.

 **Provision credentials****Uses shared OAuth credentials to authenticate with Atlan—no manual API calls.

 <strong>Configure app****Downloads artifacts, generates config from the workflow payload, and runs the configurator.

 <strong>Deploy app****On Docker: pulls the image, creates a container, and mounts Dapr components. On Kubernetes: runs `helm upgrade --install` with the bundled chart.

 <strong>Report to Atlan****Reports deployment success and updates the UI. The app is ready to use.

## Platform comparison

| Aspect | Docker / Podman | Kubernetes |
|--------|----------------|------------|
| **Runs on** | Linux VM with Docker or Podman | Any CNCF-conformant K8s cluster (EKS, GKE, AKS, k3s, etc.) |
| **App deployment** | Docker container via Unix socket | Helm release via Kubernetes API |
| **Dapr** | Runs inside the app container; components mounted as YAML files | Runs inside the app container; components delivered as ConfigMap |
| **Self-update** | Blue-green container swap (6 steps) | `helm upgrade --reuse-values` (K8s rolling update) |
| **Registry auth** | `CONTAINER_REGISTRY_USERNAME/PASSWORD` env vars | `imagePullSecrets` (K8s docker-registry secret) |
| **Port conflicts** | Yes (host port binding) | No (pod network namespace) |
| **Multi-app scaling** | Limited by single host | K8s scheduler distributes across nodes |
| **Configuration** | `config.yaml` + `.env` | `values.yaml` (Helm) |
| **Security model** | Docker socket access (docker group) | Namespace-scoped RBAC, no privileged access |
| **Best for** | Quick setup, single VM, environments without Kubernetes | Production workloads, existing K8s clusters, enterprise scale |

## Security

The SDR Orchestrator inherits the same security model as all Self-Deployed Runtime deployments—outbound-only networking (no inbound ports, no VPN tunnels), non-root execution, and OAuth 2.0 credentials shared across all apps deployed by the SDR. For full details, see [Security](https://docs.atlan.com/llms/platform/self-deployed-runtime/security/llms.txt) and [Configure network security](https://docs.atlan.com/llms/platform/self-deployed-runtime/configure-network-security/llms.txt).

On Kubernetes, the SDR additionally benefits from namespace-scoped RBAC (no cluster-wide access), Kubernetes network policies, automounted ServiceAccount tokens, and non-root pod execution (`runAsUser: 1000`).

## Platform-specific details

Pick your deployment platform below to see the runtime connection, app management, install and upgrade flow, automatic updates, design principles, and one-time setup for that platform. Kubernetes additionally includes RBAC permissions, Helm chart resources, environment variables, and supported distributions.

### Runtime connection

The orchestrator runs as a single container on your Linux VM. For container management, it connects to Docker or Podman via a local Unix socket mounted into the orchestrator container. Through this socket, it creates and manages sibling app containers—the connector apps running on the same VM. All of this communication is local; no network is involved.

### App management

#### Container management

The orchestrator manages app containers by mounting the Docker or Podman [Unix socket](https://man7.org/linux/man-pages/man7/unix.7.html) into the orchestrator container—giving it direct access to the container engine's command API without root access or elevated capabilities.

![How the SDR Orchestrator manages containers via Unix socket](https://docs.atlan.com/img/product/connections/self-deployed-runtime/orchestrator/container-management.png)

**Figure 2:** Container management via Unix socket—the SDR Orchestrator sends commands to the Docker/Podman engine through a local socket file.

The socket path depends on your container runtime:

- **Docker:** `/var/run/docker.sock`. See [Docker Engine API docs](https://docs.docker.com/engine/api/).
- **Podman (rootless):** `/run/user/{UID}/podman/podman.sock`. See [Podman system service docs](https://docs.podman.io/en/latest/markdown/podman-system-service.1.html).

Through the socket, the orchestrator sends commands to pull images, start and stop containers, and check health status. No network ports are involved—all communication between the orchestrator and the container engine is local.

#### Dapr components

Each app container runs Dapr inside the container (not as a separate sidecar). The orchestrator generates Dapr component YAML files from your `config.yaml` and mounts them into the app container at `/app/components`. Components include:

- **objectstore**:S3, GCS, or Azure Blob for data storage
- **secretstore**:AWS Secrets Manager, GCP Secret Manager, Azure Key Vault, or environment variables
- **deployment-secret-store**:reads OAuth credentials from process environment variables
- **eventstore**:HTTP binding to Atlan's event ingress endpoint

### App install and upgrade

When you install or upgrade a connector app from the Atlan UI, the following sequence takes place:

![App Deployment Flow](https://docs.atlan.com/img/product/connections/self-deployed-runtime/orchestrator/app-deployment-flow.png)

**Figure 3:** App deployment flow—user triggers install from the Atlan UI, which dispatches a Temporal workflow to the SDR.

1. **User triggers install** from the Atlan UI
2. **Marketplace API** creates a deployment record and starts a Temporal workflow on queue `atlan-sdr-<deployment-name>`
3. **SDR picks up the workflow** and deploys the app:
 - Generates Dapr component YAMLs from `config.yaml`
 - Pulls the app image from the registry
 - Creates and starts the app container with component mounts
 - Runs health checks via Docker inspect (checks container state, not HTTP)
4. On success, the SDR notifies Atlan to update the deployment records

If an app has a running workflow at the time of an upgrade, Temporal's durable execution ensures the workflow automatically resumes from where it left off once the updated app is running—no work is lost.

### Automatic update

The Atlan reconciler continuously monitors the orchestrator's version. When it detects version drift, it triggers an automatic update.

The update uses a blue-green swap—the new container is fully health-checked before the current one is replaced.

![SDR automatic update using blue-green swap](https://docs.atlan.com/img/product/connections/self-deployed-runtime/orchestrator/autoupdate-blue-green.png)

**Figure 4:** SDR auto updates using blue-green swap—a new container is created and health-checked before the old one is replaced, ensuring zero downtime.

When a newer SDR image is available:

1. Pull the new SDR image
2. Create a new container (green) with the same mounts and environment variables
3. Health-check the green container
4. Swap: rename current container to `old`, rename green to `current`
5. Notify the Atlan registry
6. Stop the old container (best-effort)

### Design principles

The SDR Orchestrator is designed around a single constraint: it must operate entirely within your security boundary without requiring any changes to it. Every design decision follows from that.

**Outbound-only networking** means the orchestrator never opens a listening port. All communication is initiated from within your infrastructure—the orchestrator reaches out to Atlan Cloud, Atlan never reaches in. There are no inbound connections to configure, no firewall rules to add, and no VPN tunnels to maintain.

**Unprivileged execution** means the orchestrator runs as a non-root user with no `--privileged` flag and no elevated kernel capabilities. It manages containers through the Unix socket without needing host-level access—the socket provides exactly the permissions it needs, no more.

:::note Docker socket access
Docker socket access grants the ability to manage all containers on the host, which is functionally equivalent to `docker` group membership. For environments requiring stronger isolation, use rootless Podman where the socket is scoped to a single non-root user.
:::

**Single entry point** means you deploy one orchestrator and it manages everything else. This keeps the operational surface small. There's only one thing to monitor, restart, or update at the infrastructure level; the orchestrator handles the rest.

**Automatic updates** means the orchestrator can update itself without manual intervention. You're never stuck on an outdated version, and updates don't interrupt running workloads.

### One-time setup

Three steps, done once. After this, you never touch the terminal for individual apps.

 <strong>Prepare your VM****Install Docker or Podman, get a Docker Hub PAT from Atlan, provision object storage and a secret store. See [system requirements](https://docs.atlan.com/llms/platform/self-deployed-runtime/install-with-orchestrator/llms.txt).

 <strong>Create config.yaml****Add Atlan credentials, storage, secret store, and registry settings.

 <strong>Start the orchestrator****Run one `podman run` or `docker run` command. The SDR connects to Atlan and waits for instructions.

### Runtime connection

The orchestrator runs as a Kubernetes Deployment (replicas=1) in the `atlan` namespace. It uses an in-cluster ServiceAccount token (automounted at `/var/run/secrets/kubernetes.io/serviceaccount/`) to access the Kubernetes API, and the Helm CLI to deploy and manage app Deployments in the same namespace. All API access is namespace-scoped via RBAC—no cluster-wide permissions are required. For RBAC details, see [RBAC permissions](#rbac-permissions).

### App management

#### Helm-based deployment

The orchestrator deploys each app using `helm upgrade --install` with a bundled Helm chart. The same chart is used by Atlan's own infrastructure, ensuring zero deployment divergence between Atlan-managed and self-deployed apps. Each app gets its own Helm release that creates:

- **Deployment**:the app pod with rolling update strategy and readiness/liveness probes
- **Service**:cluster-internal networking (ClusterIP on port 8000)
- **ServiceAccount**:app identity with optional RBAC for K8s secret access
- **ConfigMap**:Dapr component definitions (objectstore, secretstore, eventstore, deployment-secrets)

The orchestrator has a ServiceAccount with a namespaced Role granting permissions to manage these resources in the `atlan` namespace only. For the full permissions table, see [RBAC permissions](#rbac-permissions).

#### Dapr components

On Kubernetes, Dapr also runs inside the app container. The orchestrator generates Dapr component YAML files and delivers them as a ConfigMap mounted into the app container. Components include:

- **objectstore**:S3, GCS, or Azure Blob for data storage
- **secretstore**:AWS Secrets Manager, GCP Secret Manager, Azure Key Vault
- **deployment-secret-store**:reads OAuth credentials from a Kubernetes Secret
- **eventstore**:HTTP binding to Atlan's event ingress endpoint

#### Health checks

Health checks poll `deployment.status.availableReplicas` via the Kubernetes API every 3 seconds with Temporal heartbeats. The SDR also inspects pod container states and immediately fails the deployment if it detects terminal conditions:

- `CrashLoopBackOff`:container keeps crashing
- `ImagePullBackOff` / `ErrImagePull`:image registry or auth issue
- `CreateContainerConfigError`:invalid container configuration
- `InvalidImageName`:malformed image URL
- `CreateContainerError` / `RunContainerError`:runtime setup failed

#### Registry authentication

App images are pulled from a private container registry. On Kubernetes, the orchestrator uses a standard `docker-registry` Secret as an `imagePullSecret`:

```bash
kubectl create secret docker-registry atlan-registry-creds \
 --docker-server=docker.io \
 --docker-username=<username> \
 --docker-password=<pat> \
 -n atlan
```

The secret name is passed to every app Helm release as `imagePullSecrets`, so all app pods can pull private images without additional configuration.

### App install and upgrade

When you install or upgrade a connector app from the Atlan UI, the following sequence takes place:

![App Deployment Flow](https://docs.atlan.com/img/product/connections/self-deployed-runtime/orchestrator/app-deployment-flow.png)

**Figure 3:** App deployment flow—user triggers install from the Atlan UI, which dispatches a Temporal workflow to the SDR.

1. **User triggers install** from the Atlan UI
2. **Marketplace API** creates a deployment record and starts a Temporal workflow on queue `atlan-sdr-<deployment-name>`
3. **SDR picks up the workflow** and deploys the app:
 - Transforms the app config into Helm values (env vars, Dapr components, resources, replicas)
 - Runs `helm upgrade --install` with the bundled app chart
 - Monitors the Deployment rollout until all replicas are ready
4. On success, the SDR notifies Atlan to update the deployment records

If an app has a running workflow at the time of an upgrade, Temporal's durable execution ensures the workflow automatically resumes from where it left off once the updated app is running—no work is lost.

### Automatic update

The Atlan reconciler continuously monitors the orchestrator's version. When it detects version drift, it triggers an automatic update.

The update uses a Kubernetes rolling update: `helm upgrade --reuse-values` patches the SDR Deployment with the new image tag, and Kubernetes handles the rollout.

When a newer SDR image is available:

1. Atlan triggers a self-update workflow on the SDR's task queue
2. The SDR runs `helm upgrade --reuse-values` on its own Helm release with the new image tag (`maxSurge=1`, `maxUnavailable=0`)
3. Kubernetes creates a new pod before terminating the old one—ensuring zero downtime
4. The new pod starts, connects to Temporal, and resumes processing
5. The SDR notifies Atlan to update the deployment records

### Design principles

The SDR Orchestrator is designed around a single constraint: it must operate entirely within your security boundary without requiring any changes to it. Every design decision follows from that.

**Outbound-only networking** means the orchestrator never opens a listening port. All communication is initiated from within your infrastructure—the orchestrator reaches out to Atlan Cloud, Atlan never reaches in. There are no inbound connections to configure, no firewall rules to add, and no VPN tunnels to maintain.

**Least-privilege RBAC** means the orchestrator's ServiceAccount has a namespaced Role scoped to the `atlan` namespace only. It can create and manage Deployments, Services, ConfigMaps, Secrets, and RBAC resources within that namespace—nothing else. It can't access other namespaces or cluster-level resources.

**Pod security** means the orchestrator runs as non-root (`runAsUser: 1000`) with no elevated capabilities. No privileged containers, no host networking, no host PID.

**Single entry point** means you deploy one orchestrator and it manages everything else. This keeps the operational surface small. There's only one thing to monitor, restart, or update at the infrastructure level; the orchestrator handles the rest.

**Automatic updates** means the orchestrator can update itself without manual intervention. You're never stuck on an outdated version, and updates don't interrupt running workloads.

### One-time setup

Three steps, done once. After this, you never touch the terminal for individual apps.

 <strong>Prepare your cluster****Ensure `kubectl` and `helm` (v3+) are configured. Create the `atlan` namespace and a `docker-registry` secret for image pulls. See [system requirements](https://docs.atlan.com/llms/platform/self-deployed-runtime/install-orchestrator-on-kubernetes/llms.txt).

 <strong>Configure values.yaml****Add Atlan credentials, Dapr components (object store, secret store), and registry settings.

 <strong>Install via Helm**
Run `helm install` with the bundled chart. The SDR connects to Atlan and waits for instructions.

### RBAC permissions

The SDR uses a namespace-scoped **Role** (not ClusterRole) bound to its ServiceAccount. All permissions are limited to the namespace where the SDR is deployed (default: `atlan`).

| API group | Resources | Verbs | Why |
|-----------|-----------|-------|-----|
| `""` (core) | pods, services, serviceaccounts, configmaps, secrets | create, get, list, update, patch, delete, watch | Helm creates these per app; pods are read for health checks |
| `apps` | deployments, replicasets | create, get, list, update, patch, delete, watch | Helm creates app Deployments; replicasets for rollout tracking |
| `rbac.authorization.k8s.io` | roles, rolebindings | create, get, list, update, patch, delete | Helm app chart creates per-app RBAC (opt-in) |
| `autoscaling` | horizontalpodautoscalers | create, get, list, update, patch, delete | Helm app chart creates HPA (when enabled) |
| `monitoring.coreos.com` | servicemonitors | create, get, list, update, patch, delete | Prometheus ServiceMonitor (when enabled) |

The SDR never requests cluster-wide permissions. If you need to audit what the SDR can do, inspect the Role in your namespace:

```bash
kubectl get role -n atlan -l app=atlan-sdr -o yaml
```

### Helm charts

Two Helm charts are bundled inside the SDR container image:

| Chart | Path in image | Purpose | Used by |
|-------|--------------|---------|---------|
| `atlan-app` | `/app/helm/app/` | Deploy marketplace apps | `trigger_cd` activity |
| `atlan-sdr` | `/app/helm/sdr/` | Deploy/update SDR itself | Customer `helm install` + `self_update()` |

The `atlan-app` chart is the same chart used by Atlan's own infrastructure (Flux), ensuring zero deployment divergence between Atlan-managed and self-deployed apps.

#### What `helm install` creates for SDR

| Resource | Name pattern | Purpose |
|----------|-------------|---------|
| Deployment | `atlan-sdr-<name>` | SDR pod (replicas=1, maxSurge=1, maxUnavailable=0) |
| Secret | `atlan-sdr-<name>-credentials` | OAuth creds + deployment name (mounted as envFrom) |
| ConfigMap | `atlan-sdr-<name>-config` | config.yaml (domain, Dapr component specs) |
| ServiceAccount | `atlan-sdr-<name>-serviceaccount` | K8s API identity for the SDR |
| Role | `atlan-sdr-<name>-role` | Namespace-scoped permissions (see [RBAC permissions](#rbac-permissions)) |
| RoleBinding | `atlan-sdr-<name>-rolebinding` | Binds the role to the ServiceAccount |

#### What each app deployment creates

| Resource | Purpose |
|----------|---------|
| Deployment | App pod with rolling update strategy, readiness/liveness probes |
| ConfigMap | Dapr component YAMLs (objectstore, secretstore, eventstore, deployment-secrets) |
| Service | In-cluster networking (ClusterIP on port 8000) |
| ServiceAccount | App identity (opt-in RBAC for K8s secret access) |

### Environment variables

#### Configurable timeouts

| Variable | Default | Purpose |
|----------|---------|---------|
| `SDR_HELM_DEPLOY_TIMEOUT` | `360` (seconds) | Helm deploy subprocess timeout |
| `SDR_HELM_UNINSTALL_TIMEOUT` | `120` (seconds) | Helm uninstall subprocess timeout |
| `SDR_HELM_SELF_UPDATE_TIMEOUT` | `60` (seconds) | Helm self-update subprocess timeout |
| `SDR_HEALTH_CHECK_TIMEOUT` | `120` (seconds) | Health check polling timeout |

#### Runtime configuration

| Variable | Default | Purpose |
|----------|---------|---------|
| `K8S_APPS_NAMESPACE` | `atlan` (autodetected from pod namespace) | Namespace for app deployments |
| `K8S_IMAGE_PULL_SECRET` | (empty) | Registry secret name for app pods |
| `CONTAINER_REGISTRY_BASE` | (empty) | Private registry prefix for image URL rewriting |
| `SDR_CONFIG_PATH` | `/mnt/config/config.yaml` | Config file location inside the pod |
| `DAPR_LOG_LEVEL` | `warn` | Dapr log level |
| `LOG_LEVEL` | `INFO` | SDR worker log level |

Override these by setting environment variables in `values.yaml` under `sdr.credentials` or as extra env vars in the Helm chart.

### Supported Kubernetes distributions

The SDR uses standard Kubernetes APIs and works on any CNCF-conformant distribution:

| Platform | Vendor | Status |
|----------|--------|--------|
| EKS | AWS | Supported |
| GKE | Google | Supported |
| AKS | Azure | Supported |
| OpenShift | Red Hat | Supported |

## See also

- [Install SDR Orchestrator on Docker/Podman](https://docs.atlan.com/llms/platform/self-deployed-runtime/install-with-orchestrator/llms.txt): Step-by-step Docker setup guide.
- [Install SDR Orchestrator on Kubernetes](https://docs.atlan.com/llms/platform/self-deployed-runtime/install-orchestrator-on-kubernetes/llms.txt): Step-by-step Kubernetes setup guide.
- [SDR Orchestrator FAQ](https://docs.atlan.com/llms/platform/self-deployed-runtime/orchestrator-faq/llms.txt): Troubleshooting, monitoring, and supported platforms.
- [Architecture](https://docs.atlan.com/llms/platform/self-deployed-runtime/architecture/llms.txt): Core architecture for all Self-Deployed Runtime deployments.

---