Set up Cross-Connection Lineage

✦TL;DR

Generate lineage across multiple connections in the same connector using the Atlan Standard Lineage app. Covers BigQuery cross-connection lineage setup, prerequisites, configuration, and known limitations.

||

|Connect

Generate lineage across multiple connections that belong to the same connector using the Atlan Standard Lineage app. Without Standard Lineage, lineage stops at the boundary of each connection: a query in BigQuery connection A that reads from BigQuery connection B won't produce lineage spanning both connections. Standard Lineage resolves this by consuming the query history from your miners and resolving asset references across the selected connections.

BigQuery only in this release

Standard Lineage is generally available for BigQuery.

Prerequisites

Before you begin, make sure you have:

• At least two BigQuery connections in Atlan (the same workflow also generates same-connection lineage, so a single connection is valid, but cross-connection lineage requires two or more).
• A crawler configured and run for every connection you want to include. Crawlers provide the asset metadata that Standard Lineage needs to resolve table and column references.
• A miner (query log miner) configured and run for every connection you want to include. Miners extract the query history that Standard Lineage uses to build lineage relationships.
• Access to the Atlan Standard Lineage app. You can verify this by searching for Standard Lineage in the Atlan marketplace. If you don't see it, contact Atlan support or your Atlan customer team to request access.

Configure workflow

Open workflow setup

1. In your Atlan workspace, click Connectors in the left sidebar.

   If you are using the Old UI (Classic), click New Workflow in the top navigation.

2. Search for Standard Lineage and select Set up workflow.

Select connector

3. In the Connector field, select BigQuery.

One connector per workflow

All connections in a Standard Lineage workflow must belong to the same connector. You can't mix connections from different connectors (for example, BigQuery and Snowflake) in a single workflow.

Select connections

4. In the Connections field, select one or more connections to include.

   • Select all connections that participate in cross-connection data flows. If connection A writes to tables in connection B, include both connections.
   • Each connection you select must already have a crawler and a miner run against it. See Prerequisites.
   • You can add or remove connections later by editing the workflow configuration and re-running.

One workflow for all connections

Prefer one Standard Lineage workflow that contains all relevant connections over multiple narrowly scoped workflows. A single workflow resolves references across all included connections in one pass and is simpler to maintain. Adding a new connection to an existing workflow is supported without a full re-run of previously included connections.

Name standard lineage connection

5. In the Standard Lineage Connection Name field, enter a name for the internal connection asset that this workflow creates. This connection groups the lineage process assets produced across your included connections.

   For example:

   standard-lineage-bigquery-prod

Schedule (optional)

6. (Optional) Configure a Schedule to run the workflow automatically.

   • Daily or more frequent runs are recommended because Standard Lineage scales with query-history volume, not connection count.

Run workflow

7. When your configuration is complete:

   • Click Run to start the workflow immediately, or
   • Click Save to save the configuration for a scheduled or future manual run.

   The workflow status updates in the Connectors panel. A successful run shows a green check; click the run entry to see per-connection progress and any errors.

What happens after workflow runs

• Standard Lineage generates lineage for queries where the source and target assets both resolve within the included connections.
• It generates both same-connection lineage (source and target in the same connection) and cross-connection lineage (source in one connection, target in another).
• Table-level and column-level lineage are generated where the extracted query history includes sufficient column detail.
• After a connection is added to Standard Lineage, lineage generation for that connection shifts from the connection's own crawler or miner to the Standard Lineage workflow. Your crawler and miner continue to run as usual and keep metadata and query history up to date.
• Removing a connection from the workflow returns lineage generation for that connection to its own crawler or miner on the next run.
• Adding a new connection to an existing workflow doesn't require a full re-run of previously included connections. Each connection resumes from its own last processed point. The new connection must already have crawler and miner output available.

Known limitations

• Same connector only. All connections in a single workflow must belong to the same connector.

• Duplicate asset names across connections. If the same database, schema, and table identity exists in more than one included connection, Standard Lineage applies an origin-first rule: if the connection where the query ran (the origin connection) owns the referenced asset, lineage is attributed to that connection only. If the origin connection doesn't own the asset, lineage is attributed to every included connection that does. This means a table present in both a staging and a production connection may receive lineage edges from both when the query originated outside those connections.

Need help?

If you have any issues configuring the app, contact Atlan support.

See also

• Generate lineage between assets: Create lineage between assets across different systems by matching asset names or patterns, a different use case from query-history-based cross-connection stitching.
• Capture and troubleshoot lineage: Best practices for reliable lineage capture, including mechanisms available for cross-system and cross-connection use cases.