Skip to main content

Run Standard Lineage app
App

TL;DR

Build lineage across multiple connections of a single connector using the Python SDK, and re-scope the workflow as connections are onboarded.

Connect

You can use the Standard Lineage app to build lineage across multiple connections of the same connector. For example, several BigQuery connections that reference each other. It reads the query history each connection's own miner already extracted, so it doesn't crawl a source or take a credential. This guide shows how to configure, run, and re-scope the app with the Python SDK (pyatlan).

One workflow owns a set of connections; onboarding a new connection means adding it to that set.

Prerequisites​

Before you begin:

  • Make sure the Standard Lineage app is available on your tenant's marketplace. If you can't see it, raise a support request to have it enabled.
  • Set up the Python SDK and configure ATLAN_BASE_URL and ATLAN_API_KEY.
  • Crawl and mine each connection you want in scope first—Standard Lineage reads the query history their miners produced. All connections in one workflow must belong to the same connector.

Configure app workflow​

Build the workflow with the AtlanStandardLineage builder. Pass a name for the workflow's own connection and the source connections to build lineage across—no credential is needed.

Configure the cross-connection scope
from pyatlan.client.atlan import AtlanClient
from pyatlan.model.apps import AtlanStandardLineage

client = AtlanClient()

lineage = (
    AtlanStandardLineage(client)
    .connection(name="bigquery-cross-connection")  # the workflow's own connection
    .connections([                                  # source connections, same connector
        "default/bigquery/1700000000",
        "default/bigquery/1700000001",
    ])
)
  • Connection names the workflow's own connection; the SDK mints it under the standard-lineage connector.
  • Connections is the scope—the source connection qualified names to build lineage across. They must all belong to one connector (the connector is derived from them; pass connector= to override). An empty scope, a mixed-connector scope, or the workflow's own connection in the scope is rejected.

Run app workflow​

Submit the run with a unique name:

Run the lineage build
response = lineage.run(name="bigquery cross-connection lineage")
print(response.slug, response.run_id)

.run(...) creates the workflow and submits a run; use .create(...) to create it without running.

Update existing workflow​

Publish a new version on the same slug with the builder's load() / update(), the same flow as every app builder. It reads the workflow's current inputs, lets you change only the fields you pass, and preserves everything else (here, the workflow's own connection and its admins). Then submit a run to apply it.

Update via the builder, then re-run
from pyatlan.model.apps import AtlanStandardLineage

# Change only the scope; the own connection and admins are preserved.
AtlanStandardLineage(client).load("bigquery-cross-connection-AbC123").connections([
    "default/bigquery/1700000000",
    "default/bigquery/1700000001",
]).update()

client.app.submit("bigquery-cross-connection-AbC123")  # re-run to apply

For onboarding, prefer the delta helpers—they compute the new scope from the current one and are idempotent (a no-op publishes no new version):

Add, remove, and read the scope
lineage = AtlanStandardLineage(client)
lineage.add_connections("bigquery-cross-connection-AbC123", ["default/bigquery/1700000002"])    # onboard one
lineage.remove_connections("bigquery-cross-connection-AbC123", ["default/bigquery/1700000002"]) # remove one, keep the rest
print(lineage.get_connections("bigquery-cross-connection-AbC123"))                              # read the current scope

Removing every connection is refused—the app can't run on an empty scope. To stop processing everything, delete the workflow instead.

See Manage apps for the rest of the lifecycle—list, get, run status, schedules, and delete—shared by every app.

Validate results​

After the run starts:

  • Open the app's Runs tab in Atlan and confirm the run completes.
  • Check the cross-connection lineage in the lineage graph for assets in the scoped connections.

To check run status programmatically, pass response.run_id to client.app.get_run(...).

Need help​

If the app isn't available on your tenant or a run fails, contact Atlan Support by submitting a request.

See also​

  • Manage apps: create, list, re-run, schedule, and delete app workflows.