Skip to main content

Configure workflow for SDR mode of execution

When using Self-deployed Runtime mode of execution, source system credentials (secrets) are retrieved from the secret store configured as part of the SDR installation. This guide provides steps to configure workflows with Self-deployed Runtime and specify the secret details it uses during workflow execution.

Before you begin​

  • An active Self-deployed Runtime instance on your Atlan tenant
  • Your infrastructure team configured which secret store the SDR connects to during installation.
    • Depending on whether the required secrets have already been created, you need either knowledge of the secret paths/keys or read/write access to the secret store itself.

Set up SDR mode of execution​

When creating or editing a connector workflow in the Atlan UI:

  1. Set Extraction Method: Select Self-Deployed Runtime as the extraction method. This reveals two additional fields: Agent and Key Type.
  2. Select your SDR instance: From the Agent dropdown, select the Self-Deployed Runtime instance you want to run this workflow on.
info

The Agent field is being renamed to Self-Deployed Runtime in an upcoming release.

Configure secrets for workflow execution​

Determine key type​

The secret store you configured during SDR installation determines whether you use Multi-Key or Single-Key references in your workflow configuration. The difference comes down to how your secret store organizes entries.

  • Multi-Key stores (for example, AWS Secrets Manager, HashiCorp Vault, Kubernetes Secrets) support storing multiple key-value pairs in a single secret entry. You can store all credentials for a workflow in one entry and reference individual keys for each field.
  • Single-Key stores (for example, GCP Secret Manager, Azure Key Vault, environment variables) map each entry to a single value. Each workflow field references a separate entry in the secret store.

Store sensitive fields in secret vault/manager​

💪 Did you know?

You decide which fields in a workflow configuration are sensitive and which aren't. Sensitive fields reference a secret stored in your vault, while non-sensitive fields can be specified directly in the workflow configuration.

At least one field (for example, password, token, or API key) must be stored as a secret.

Before configuring the workflow in Atlan, store the required credentials in your secret vault or manager. How you organize the entries depends on whether your store is Multi-Key or Single-Key.

Example: Oracle connector workflow​

Consider an Oracle workflow and you consider the following information to be sensitive: Host, Port, Username, and Password.

With a Multi-Key store, you create one top-level entry in the store, for example, Atlan/Workflows/Oracle/Oracle_UAT_DWH, and this entry contains the following key-value pairs:

{
  "db_host": "oracle-db.example.com",
  "db_port": "1521",
  "db_user": "ATLAN_USER",
  "db_password": "s3cretP@ss"
}

Configure secret references in workflow configuration​

Follow these steps in the Atlan UI when you set up or edit connector workflows. You decide which fields are sensitive (referenced from the secret store) and which are non-sensitive (entered directly).

  1. Select Key Type: Set the Key Type toggle to Multi Key. This reveals a new field: Secret Path/Name in Secret Manager.

  2. Set the secret path: Enter the top-level secret entry name in Secret Path/Name in Secret Manager: Atlan/Workflows/Oracle/Oracle_UAT_DWH.

  3. Sensitive fields: For fields that reference secrets, enter only the key from the secret entry:

    Workflow fieldValue (secret reference)
    Hostdb_host
    Portdb_port
    Usernamedb_user
    Passworddb_password

  4. Non-sensitive fields: For fields that don't contain sensitive information, enter the actual value directly:

    Workflow fieldValue (actual value)
    Service NameORCL
    Database NameUAT_DWH
What happens at runtime?

When the workflow runs, the SDR component running in your infrastructure authenticates to the secret store, fetches the secret values using the references you provided in the workflow configuration, and uses them to connect to the source system.

Configure file inputs for workflow execution​

Some connector workflows need file inputs in addition to credentials, for example, a Kerberos keytab, a krb5.conf, or a TLS certificate. In SDR mode, each file field accepts three input forms, and you can pick a different form per file.

Atlan stays flexible, you stay in control

Organizations differ on which files are acceptable to upload to SaaS, store in a secret manager, or place in object storage. SDR supports all three so you can match each file to your own policy.

Three input forms​

  1. Upload via the Atlan UI. Click the upload icon next to the field and select the file. The file is stored on your Atlan tenant and delivered to SDR at workflow run time. Best for small, one-off files.

  2. Reference a key in your secret store. Enter the key name in the secret store you configured for the SDR (for example, atlan/workflows/hive/keytab). The value at that key must be the base64-encoded contents of the file. At run time, SDR fetches the value via Dapr, decodes it, and provides it to the workflow.

    Most secret stores cap a single entry's value around 64 KB. For files that don't fit, use the object-store option described next.

  3. Reference a path in object storage. Enter the path using the objectstore:// scheme. The text after objectstore:// is the object key inside the bucket configured in your objectstore Dapr binding (set when you installed the SDR). At run time, SDR fetches the file from that location via Dapr.

Example: Object store path resolution​

Say your objectstore binding points to an AWS S3 bucket:

# objectstore.yaml: points at the customer's own S3 bucket
spec:
  type: bindings.aws.s3
  metadata:
    - name: bucket
      value: acmebank-atlan-sdr

You've placed your Hive credential files under a folder of your choosing inside that bucket:

s3://acmebank-atlan-sdr/
└── atlan-credentials/
    └── hive/
        ├── krb5.conf
        └── certificate.cer

In each connector workflow file field, type the object key (the path inside the bucket):

FieldValue to enter
Kerberos Config File (krb5.conf)objectstore://atlan-credentials/hive/krb5.conf
CA Certificate Fileobjectstore://atlan-credentials/hive/certificate.cer

At run time, SDR resolves each objectstore://<key> to s3://acmebank-atlan-sdr/<key> and fetches the file via Dapr. The same pattern applies to GCS, Azure Blob, or any other Dapr-supported object store: the bucket or container comes from the binding, and the key comes from the workflow field.

How fields detect your input​

Each file field is a single text input with an upload icon. SDR routes based on what it receives. There's no separate toggle to set:

You provideSDR resolves it as
An uploaded fileThe file delivered from the Atlan tenant
Text starting with objectstore://A path in your object-store binding
Any other textA key in your secret store

You can mix forms freely within a single workflow. For example, on a Hive mTLS connection, you might upload the CA certificate via the Atlan UI, point at the client certificate with an objectstore:// path, and reference the client key as a key in your secret store. Each file lands in the storage that best fits its sensitivity and size.

If the value you provide doesn't match any of the three forms, or if SDR can't resolve it at run time (for example, the referenced secret-store key or objectstore:// path doesn't exist), the workflow run fails with an error.

Next steps​

After configuring the Self-deployed Runtime for workflow execution, return to your connector's setup guide and continue the workflow setup.