Skip to main content

Migrate to Self-deployed runtime

This guide walks you through migrating your metadata extraction workflows from Secure Agent to Self-Deployed Runtime (SDR). Most teams complete the migration in a few hours per connector.

Before you begin, read Understand migrating to SDR to choose a deployment mode and understand what changes. Use the configuration mapping to look up how your existing settings translate to SDR.

Plan migration​

Before deploying SDR, complete the following to prepare your environment and gather what you need.

  1. Choose deployment mode: Decide between SDR Orchestrator and SDR Single App. SDR Orchestrator is recommended—it manages all connector apps from the Atlan UI with automated lifecycle management. Choose SDR Single App if you are migrating 1–2 connectors and prefer manually updating images. See Deployment options for a full comparison.

  2. Open firewall for gRPC (Infrastructure / Network Engineer): Allow outbound gRPC/TLS from the SDR host to <tenant>-temporal.atlan.com:443. Verify from the target host:

    curl -v https://<tenant>-temporal.atlan.com:443

    See Configure network security for detailed firewall rules.

  3. Generate OAuth credentials (Atlan Admin): SDR uses per-deployment OAuth credentials instead of a shared API key. In the Atlan UI, create an API token, then register your deployment:

    curl -X POST "https://<tenant>.atlan.com/api/sdr/auth/register" \
      -H "Authorization: Bearer YOUR_API_TOKEN" \
      -H "Content-Type: application/json" \
      -d '{"deploymentName": "YOUR_DEPLOYMENT_NAME"}'

    Save the clientId and clientSecret from the response—you'll need them in the SDR .env file.

  4. Obtain Docker Hub PAT (Atlan representative): Contact your Atlan representative for the Docker Hub PAT and authenticate on the target host:

    docker login -u atlanhq
    # Enter the PAT when prompted

    The PAT rotates periodically. If you embed it in CI/CD pipelines, update those jobs when the PAT changes.

  5. Update CI/CD pipelines (DevOps, if applicable): If existing pipelines pull Secure Agent container images, update them to authenticate with Docker Hub using the PAT and pull from docker.io/atlanhq/*.

  6. Inventory Secure Agent: Document every workflow currently running on your Secure Agent. This becomes your checklist for Post-migration. In Workflow Center, filter by execution mode to show only Secure Agent workflows and record for each:

    • Source system and connector type
    • Schedule
    • Secret store references (vault path or key names—not the secrets themselves)
    • Object storage configuration
    • Secure Agent name and deployment type (K3s or EKS)

Migrate to SDR​

With planning complete, deploy SDR and move your workflows one at a time.

  1. Deploy SDR: Follow the platform-specific install guide for your chosen deployment mode. It covers all configuration, startup, and verification steps. See Install Self-Deployed Runtime for detailed instructions.

  2. Install connector apps (Orchestrator only): For each connector type in your inventory, install the corresponding SDR app from the Atlan UI Marketplace. Each Secure Agent connector maps 1:1 to an SDR app—if you run Snowflake and Tableau workflows, install the Snowflake app and the Tableau app. See Manage apps on SDR. Single App deployments: skip this step—you already selected the connector image during deployment.

  3. Switch workflow configuration: For each workflow in your inventory, go to Workflow Center, open the workflow, change the Agent dropdown from your Secure Agent to your SDR deployment, and save.

  4. Run the workflow: Trigger a manual run for each switched workflow in Workflow Center and monitor until completion.

Post-migration validations​

Once all workflows have been switched and run on SDR, validate the results and confirm readiness before decommissioning Secure Agent.

warning

Keep Secure Agent running until all workflows have completed at least 1–2 full extraction cycles on SDR. If any workflow fails, revert it to Secure Agent while troubleshooting before decommissioning.

  1. Validate extracted assets: Compare results with the last Secure Agent extraction. Confirm that metadata extraction produces expected asset counts, the same asset types (tables, views, columns, and other entity types) appear, key lineage paths are intact, and glossary tags and term links persist.

  2. Confirm no workflows reference Secure Agent: In Workflow Center, filter by execution mode and verify no workflows still point to the old Secure Agent.

Uninstall Secure Agent​

Once all checklist items pass, remove Secure Agent from your infrastructure. Select your deployment type:

  1. Stop and disable the rootless K3s service:

    systemctl --user stop k3s-rootless
    systemctl --user disable k3s-rootless

  2. Remove the service unit and reload:

    rm -f ~/.config/systemd/user/k3s-rootless.service
    systemctl --user daemon-reload

  3. Remove the Secure Agent installation directory:

    rm -rf ~/atlan-secure-agent

Clean up Cloud resources​

If you changed your object storage configuration during migration, remove artifacts from the old location. If you pointed SDR at the same bucket as Secure Agent, skip this step.

  • AWS S3: remove old extraction artifacts from the Secure Agent bucket
  • Azure Blob Storage: remove old containers or blobs
  • GCS: remove old extraction objects

See also​