LINEAGE_ADJACENCY_LIST table
The LINEAGE_ADJACENCY_LIST table is a flat list of directed lineage edges between assets and processes. Each row represents a single hop, where from_guid is the upstream asset or process and to_guid is the downstream asset or process.
Use this table to traverse lineage starting from any asset. Pair it with the ASSETS table to resolve GUIDs to human-readable names and types.
Columns
The LINEAGE_ADJACENCY_LIST table includes the following columns:
| Column | Description |
|---|---|
from_guid | The GUID of the upstream asset or process in the edge |
to_guid | The GUID of the downstream asset or process in the edge |
Example queries
The examples below cover Snowflake, Databricks, and BigQuery. Each engine's tab includes a direct multi-hop query and a cycle-detection variant that's safe for graphs that may contain cycles.
Replace the placeholder values for your platform:
- Snowflake: examples assume you've set your default database and schema (for example,
USE DATABASE ATLAN_CONTEXT_STORE). Otherwise, fully qualify each table reference (for example,ATLAN_CONTEXT_STORE."gold".lineage_adjacency_list). - Databricks: replace
LAKEHOUSE_CATALOGwith your Lakehouse catalog name in Unity Catalog. Recursive CTEs require Databricks Runtime 14.0+ or a Databricks SQL warehouse on a recent channel. - BigQuery: replace
PROJECT_ID.LAKEHOUSE_DATASETwith the project and dataset where your Lakehouse external tables are registered.
For multi-hop queries that return direction, hop level, and asset names in a single result set, see Set up lineage tables for the optional LINEAGE helper view.
Upstream lineage
Find every asset upstream of a target asset, joined to ASSETS for names and types.
- Snowflake
- Databricks
- BigQuery
Direct
-- Replace '<base_guid>' with the GUID of the asset you're tracing from.
SELECT guid, asset_name, asset_type
FROM (
WITH RECURSIVE lineage AS (
SELECT from_guid, 1 AS level
FROM gold.lineage_adjacency_list
WHERE to_guid = '<base_guid>'
UNION ALL
SELECT a.from_guid, l.level + 1
FROM lineage l
JOIN gold.lineage_adjacency_list a
ON l.from_guid = a.to_guid
WHERE l.level < 50
)
SELECT DISTINCT from_guid FROM lineage
) t
LEFT JOIN gold.assets ON from_guid = guid;
With cycle detection
For graphs that may contain cycles, track visited GUIDs and skip any edge that revisits one. The hop limit is raised to 100 since cycles are filtered out.
SELECT guid, asset_name, asset_type
FROM (
WITH RECURSIVE lineage AS (
SELECT from_guid, 1 AS level, ARRAY_CONSTRUCT(from_guid) AS visited
FROM gold.lineage_adjacency_list
WHERE to_guid = '<base_guid>'
UNION ALL
SELECT a.from_guid, l.level + 1, ARRAY_APPEND(l.visited, a.from_guid)
FROM lineage l
JOIN gold.lineage_adjacency_list a
ON l.from_guid = a.to_guid
WHERE l.level < 100
AND NOT ARRAY_CONTAINS(a.from_guid::VARIANT, l.visited)
)
SELECT DISTINCT from_guid FROM lineage
) t
LEFT JOIN gold.assets ON from_guid = guid;
Direct
-- Replace '<base_guid>' with the GUID of the asset you're tracing from.
SELECT guid, asset_name, asset_type
FROM (
WITH RECURSIVE lineage AS (
SELECT from_guid, 1 AS level
FROM LAKEHOUSE_CATALOG.gold.lineage_adjacency_list
WHERE to_guid = '<base_guid>'
UNION ALL
SELECT a.from_guid, l.level + 1
FROM lineage l
JOIN LAKEHOUSE_CATALOG.gold.lineage_adjacency_list a
ON l.from_guid = a.to_guid
WHERE l.level < 50
)
SELECT DISTINCT from_guid FROM lineage
) t
LEFT JOIN LAKEHOUSE_CATALOG.gold.assets ON from_guid = guid;
With cycle detection
SELECT guid, asset_name, asset_type
FROM (
WITH RECURSIVE lineage AS (
SELECT from_guid, 1 AS level, array(from_guid) AS visited
FROM LAKEHOUSE_CATALOG.gold.lineage_adjacency_list
WHERE to_guid = '<base_guid>'
UNION ALL
SELECT a.from_guid, l.level + 1, array_append(l.visited, a.from_guid)
FROM lineage l
JOIN LAKEHOUSE_CATALOG.gold.lineage_adjacency_list a
ON l.from_guid = a.to_guid
WHERE l.level < 100
AND NOT array_contains(l.visited, a.from_guid)
)
SELECT DISTINCT from_guid FROM lineage
) t
LEFT JOIN LAKEHOUSE_CATALOG.gold.assets ON from_guid = guid;
Direct
-- Replace '<base_guid>' with the GUID of the asset you're tracing from.
SELECT guid, asset_name, asset_type
FROM (
WITH RECURSIVE lineage AS (
SELECT from_guid, 1 AS level
FROM `PROJECT_ID.LAKEHOUSE_DATASET.lineage_adjacency_list`
WHERE to_guid = '<base_guid>'
UNION ALL
SELECT a.from_guid, l.level + 1
FROM lineage l
JOIN `PROJECT_ID.LAKEHOUSE_DATASET.lineage_adjacency_list` a
ON l.from_guid = a.to_guid
WHERE l.level < 50
)
SELECT DISTINCT from_guid FROM lineage
) t
LEFT JOIN `PROJECT_ID.LAKEHOUSE_DATASET.assets` ON from_guid = guid;
With cycle detection
SELECT guid, asset_name, asset_type
FROM (
WITH RECURSIVE lineage AS (
SELECT from_guid, 1 AS level, [from_guid] AS visited
FROM `PROJECT_ID.LAKEHOUSE_DATASET.lineage_adjacency_list`
WHERE to_guid = '<base_guid>'
UNION ALL
SELECT a.from_guid, l.level + 1, ARRAY_CONCAT(l.visited, [a.from_guid])
FROM lineage l
JOIN `PROJECT_ID.LAKEHOUSE_DATASET.lineage_adjacency_list` a
ON l.from_guid = a.to_guid
WHERE l.level < 100
AND a.from_guid NOT IN UNNEST(l.visited)
)
SELECT DISTINCT from_guid FROM lineage
) t
LEFT JOIN `PROJECT_ID.LAKEHOUSE_DATASET.assets` ON from_guid = guid;
Downstream lineage
Find every asset downstream of a source asset, joined to ASSETS for names and types.
- Snowflake
- Databricks
- BigQuery
Direct
-- Replace '<base_guid>' with the GUID of the asset you're tracing from.
SELECT guid, asset_name, asset_type
FROM (
WITH RECURSIVE lineage AS (
SELECT to_guid, 1 AS level
FROM gold.lineage_adjacency_list
WHERE from_guid = '<base_guid>'
UNION ALL
SELECT a.to_guid, l.level + 1
FROM lineage l
JOIN gold.lineage_adjacency_list a
ON l.to_guid = a.from_guid
WHERE l.level < 50
)
SELECT DISTINCT to_guid FROM lineage
) t
LEFT JOIN gold.assets ON to_guid = guid;
With cycle detection
SELECT guid, asset_name, asset_type
FROM (
WITH RECURSIVE lineage AS (
SELECT to_guid, 1 AS level, ARRAY_CONSTRUCT(to_guid) AS visited
FROM gold.lineage_adjacency_list
WHERE from_guid = '<base_guid>'
UNION ALL
SELECT a.to_guid, l.level + 1, ARRAY_APPEND(l.visited, a.to_guid)
FROM lineage l
JOIN gold.lineage_adjacency_list a
ON l.to_guid = a.from_guid
WHERE l.level < 100
AND NOT ARRAY_CONTAINS(a.to_guid::VARIANT, l.visited)
)
SELECT DISTINCT to_guid FROM lineage
) t
LEFT JOIN gold.assets ON to_guid = guid;
Direct
-- Replace '<base_guid>' with the GUID of the asset you're tracing from.
SELECT guid, asset_name, asset_type
FROM (
WITH RECURSIVE lineage AS (
SELECT to_guid, 1 AS level
FROM LAKEHOUSE_CATALOG.gold.lineage_adjacency_list
WHERE from_guid = '<base_guid>'
UNION ALL
SELECT a.to_guid, l.level + 1
FROM lineage l
JOIN LAKEHOUSE_CATALOG.gold.lineage_adjacency_list a
ON l.to_guid = a.from_guid
WHERE l.level < 50
)
SELECT DISTINCT to_guid FROM lineage
) t
LEFT JOIN LAKEHOUSE_CATALOG.gold.assets ON to_guid = guid;
With cycle detection
SELECT guid, asset_name, asset_type
FROM (
WITH RECURSIVE lineage AS (
SELECT to_guid, 1 AS level, array(to_guid) AS visited
FROM LAKEHOUSE_CATALOG.gold.lineage_adjacency_list
WHERE from_guid = '<base_guid>'
UNION ALL
SELECT a.to_guid, l.level + 1, array_append(l.visited, a.to_guid)
FROM lineage l
JOIN LAKEHOUSE_CATALOG.gold.lineage_adjacency_list a
ON l.to_guid = a.from_guid
WHERE l.level < 100
AND NOT array_contains(l.visited, a.to_guid)
)
SELECT DISTINCT to_guid FROM lineage
) t
LEFT JOIN LAKEHOUSE_CATALOG.gold.assets ON to_guid = guid;
Direct
-- Replace '<base_guid>' with the GUID of the asset you're tracing from.
SELECT guid, asset_name, asset_type
FROM (
WITH RECURSIVE lineage AS (
SELECT to_guid, 1 AS level
FROM `PROJECT_ID.LAKEHOUSE_DATASET.lineage_adjacency_list`
WHERE from_guid = '<base_guid>'
UNION ALL
SELECT a.to_guid, l.level + 1
FROM lineage l
JOIN `PROJECT_ID.LAKEHOUSE_DATASET.lineage_adjacency_list` a
ON l.to_guid = a.from_guid
WHERE l.level < 50
)
SELECT DISTINCT to_guid FROM lineage
) t
LEFT JOIN `PROJECT_ID.LAKEHOUSE_DATASET.assets` ON to_guid = guid;
With cycle detection
SELECT guid, asset_name, asset_type
FROM (
WITH RECURSIVE lineage AS (
SELECT to_guid, 1 AS level, [to_guid] AS visited
FROM `PROJECT_ID.LAKEHOUSE_DATASET.lineage_adjacency_list`
WHERE from_guid = '<base_guid>'
UNION ALL
SELECT a.to_guid, l.level + 1, ARRAY_CONCAT(l.visited, [a.to_guid])
FROM lineage l
JOIN `PROJECT_ID.LAKEHOUSE_DATASET.lineage_adjacency_list` a
ON l.to_guid = a.from_guid
WHERE l.level < 100
AND a.to_guid NOT IN UNNEST(l.visited)
)
SELECT DISTINCT to_guid FROM lineage
) t
LEFT JOIN `PROJECT_ID.LAKEHOUSE_DATASET.assets` ON to_guid = guid;
Optional: Full-lineage helper
If you frequently explore lineage from a single starting asset, the helpers below walk both directions in one call. They return edges (not just nodes) annotated with direction (upstream or downstream) and level (hop distance), and join both ends to ASSETS for names and types.
Each helper walks up to 100 hops in each direction. Cycle suppression isn't built in—use the cycle-detection variants from Example queries if your graph contains cycles.
- Snowflake
- Databricks
- BigQuery
A Snowflake stored procedure that returns a result set.
-- Replace:
-- <CONTEXT_STORE_DB> with your context store database
-- <CUSTOM_SCHEMA> with the schema where you want the procedure to live
CREATE OR REPLACE PROCEDURE <CONTEXT_STORE_DB>.<CUSTOM_SCHEMA>.FULL_LINEAGE(BASE_GUID VARCHAR)
RETURNS TABLE (
direction VARCHAR,
from_guid VARCHAR,
from_asset_name VARCHAR,
from_asset_type VARCHAR,
to_guid VARCHAR,
to_asset_name VARCHAR,
to_asset_type VARCHAR,
level INT
)
LANGUAGE SQL
EXECUTE AS CALLER
AS
$$
DECLARE
res RESULTSET DEFAULT (
WITH RECURSIVE upstream AS (
SELECT
from_guid,
to_guid,
1 AS level,
'upstream' AS direction
FROM <CONTEXT_STORE_DB>.GOLD.LINEAGE_ADJACENCY_LIST
WHERE to_guid = :BASE_GUID
UNION ALL
SELECT
a.from_guid,
a.to_guid,
l.level + 1,
'upstream'
FROM upstream l
JOIN <CONTEXT_STORE_DB>.GOLD.LINEAGE_ADJACENCY_LIST a
ON l.from_guid = a.to_guid
WHERE l.level < 100
),
downstream AS (
SELECT
from_guid,
to_guid,
1 AS level,
'downstream' AS direction
FROM <CONTEXT_STORE_DB>.GOLD.LINEAGE_ADJACENCY_LIST
WHERE from_guid = :BASE_GUID
UNION ALL
SELECT
a.from_guid,
a.to_guid,
l.level + 1,
'downstream'
FROM downstream l
JOIN <CONTEXT_STORE_DB>.GOLD.LINEAGE_ADJACENCY_LIST a
ON l.to_guid = a.from_guid
WHERE l.level < 100
),
combined AS (
SELECT
from_guid,
to_guid,
direction,
MIN(level) AS level
FROM (
SELECT * FROM upstream
UNION ALL
SELECT * FROM downstream
)
GROUP BY from_guid, to_guid, direction
)
SELECT
d.direction,
d.from_guid,
src.asset_name AS from_asset_name,
src.asset_type AS from_asset_type,
d.to_guid,
tgt.asset_name AS to_asset_name,
tgt.asset_type AS to_asset_type,
d.level
FROM combined d
LEFT JOIN <CONTEXT_STORE_DB>.GOLD.ASSETS src ON d.from_guid = src.guid
LEFT JOIN <CONTEXT_STORE_DB>.GOLD.ASSETS tgt ON d.to_guid = tgt.guid
ORDER BY d.direction, d.level
);
BEGIN
RETURN TABLE(res);
END;
$$;
Call it with the GUID of the asset you want to explore:
CALL <CONTEXT_STORE_DB>.<CUSTOM_SCHEMA>.FULL_LINEAGE('<base_guid>');
A Databricks SQL table-valued function. Requires Databricks Runtime 14.0+ (or a Databricks SQL warehouse on a recent channel) for both WITH RECURSIVE and table-returning UDFs.
-- Replace:
-- LAKEHOUSE_CATALOG with your Lakehouse catalog name
-- <CUSTOM_CATALOG> with the catalog where you want the function to live
-- <CUSTOM_SCHEMA> with the schema where you want the function to live
CREATE OR REPLACE FUNCTION <CUSTOM_CATALOG>.<CUSTOM_SCHEMA>.full_lineage(BASE_GUID STRING)
RETURNS TABLE (
direction STRING,
from_guid STRING,
from_asset_name STRING,
from_asset_type STRING,
to_guid STRING,
to_asset_name STRING,
to_asset_type STRING,
level INT
)
RETURN
WITH RECURSIVE upstream AS (
SELECT
from_guid,
to_guid,
1 AS level,
'upstream' AS direction
FROM LAKEHOUSE_CATALOG.gold.lineage_adjacency_list
WHERE to_guid = full_lineage.BASE_GUID
UNION ALL
SELECT
a.from_guid,
a.to_guid,
l.level + 1,
'upstream'
FROM upstream l
JOIN LAKEHOUSE_CATALOG.gold.lineage_adjacency_list a
ON l.from_guid = a.to_guid
WHERE l.level < 100
),
downstream AS (
SELECT
from_guid,
to_guid,
1 AS level,
'downstream' AS direction
FROM LAKEHOUSE_CATALOG.gold.lineage_adjacency_list
WHERE from_guid = full_lineage.BASE_GUID
UNION ALL
SELECT
a.from_guid,
a.to_guid,
l.level + 1,
'downstream'
FROM downstream l
JOIN LAKEHOUSE_CATALOG.gold.lineage_adjacency_list a
ON l.to_guid = a.from_guid
WHERE l.level < 100
),
combined AS (
SELECT
from_guid,
to_guid,
direction,
MIN(level) AS level
FROM (
SELECT * FROM upstream
UNION ALL
SELECT * FROM downstream
)
GROUP BY from_guid, to_guid, direction
)
SELECT
d.direction,
d.from_guid,
src.asset_name AS from_asset_name,
src.asset_type AS from_asset_type,
d.to_guid,
tgt.asset_name AS to_asset_name,
tgt.asset_type AS to_asset_type,
d.level
FROM combined d
LEFT JOIN LAKEHOUSE_CATALOG.gold.assets src ON d.from_guid = src.guid
LEFT JOIN LAKEHOUSE_CATALOG.gold.assets tgt ON d.to_guid = tgt.guid
ORDER BY d.direction, d.level;
Call it with the GUID of the asset you want to explore:
SELECT * FROM <CUSTOM_CATALOG>.<CUSTOM_SCHEMA>.full_lineage('<base_guid>');
A BigQuery stored procedure. The unbound SELECT at the end of the procedure body returns rows to the caller.
-- Replace:
-- PROJECT_ID, LAKEHOUSE_DATASET with your Lakehouse project and dataset
-- <CUSTOM_DATASET> with the dataset where you want the procedure to live
CREATE OR REPLACE PROCEDURE `PROJECT_ID.<CUSTOM_DATASET>.full_lineage`(BASE_GUID STRING)
BEGIN
WITH RECURSIVE upstream AS (
SELECT
from_guid,
to_guid,
1 AS level,
'upstream' AS direction
FROM `PROJECT_ID.LAKEHOUSE_DATASET.lineage_adjacency_list`
WHERE to_guid = BASE_GUID
UNION ALL
SELECT
a.from_guid,
a.to_guid,
l.level + 1,
'upstream'
FROM upstream l
JOIN `PROJECT_ID.LAKEHOUSE_DATASET.lineage_adjacency_list` a
ON l.from_guid = a.to_guid
WHERE l.level < 100
),
downstream AS (
SELECT
from_guid,
to_guid,
1 AS level,
'downstream' AS direction
FROM `PROJECT_ID.LAKEHOUSE_DATASET.lineage_adjacency_list`
WHERE from_guid = BASE_GUID
UNION ALL
SELECT
a.from_guid,
a.to_guid,
l.level + 1,
'downstream'
FROM downstream l
JOIN `PROJECT_ID.LAKEHOUSE_DATASET.lineage_adjacency_list` a
ON l.to_guid = a.from_guid
WHERE l.level < 100
),
combined AS (
SELECT
from_guid,
to_guid,
direction,
MIN(level) AS level
FROM (
SELECT * FROM upstream
UNION ALL
SELECT * FROM downstream
)
GROUP BY from_guid, to_guid, direction
)
SELECT
d.direction,
d.from_guid,
src.asset_name AS from_asset_name,
src.asset_type AS from_asset_type,
d.to_guid,
tgt.asset_name AS to_asset_name,
tgt.asset_type AS to_asset_type,
d.level
FROM combined d
LEFT JOIN `PROJECT_ID.LAKEHOUSE_DATASET.assets` src ON d.from_guid = src.guid
LEFT JOIN `PROJECT_ID.LAKEHOUSE_DATASET.assets` tgt ON d.to_guid = tgt.guid
ORDER BY d.direction, d.level;
END;
Call it with the GUID of the asset you want to explore:
CALL `PROJECT_ID.<CUSTOM_DATASET>.full_lineage`('<base_guid>');
See also
- Gold namespace
- ASSETS table
- Set up lineage tables for optional helper views with direction and asset names
- Lineage use cases
- Atlan metamodel reference