Skip to main content
Community Hub

Query redaction for Cloud SQL for PostgreSQL

TL;DR

Reference for Redact SQL Text on Cloud SQL for PostgreSQL crawler and miner workflows: what Atlan redacts, what it preserves, and limitations to plan for.

Your AI can read this via Docs MCPcurl -fsSL "https://docs.atlan.com/install-docs-mcp" | bashConnect

By default, Atlan stores the SQL text it extracts from Cloud SQL for PostgreSQL exactly as written at source. That text can contain literal values from your business data, such as email addresses in a WHERE clause or account numbers in a view definition.

Query redaction replaces those literal values with placeholders before Atlan stores them. Redaction runs in memory while the workflow extracts data, so the original values aren't written to Atlan's storage or sent outside your environment. Query structure, table references, and column references are preserved, so lineage is unaffected.

Redaction is off by default. You opt in per workflow using the Redact SQL Text option.

Enable redaction​

Redact SQL Text is a separate option on each workflow, so you can enable it for the crawler, the miner, or both. The miner doesn't inherit the crawler's setting, so enable it on each workflow whose SQL text you want redacted.

WorkflowWhere to find it
Cloud SQL for PostgreSQL crawlerConfigure crawler screen
Cloud SQL for PostgreSQL minerConfigure miner screen

The option appears for both direct and Self-Deployed Runtime extraction. With Self-Deployed Runtime extraction, redaction runs inside your Self-Deployed Runtime.

What Atlan redacts​

Redaction applies to the fields that carry SQL text for each workflow.

WorkflowRedacted content
CrawlerView and materialized view definitions, and the bodies of functions and procedures
MinerQuery text from pg_stat_statements

Within that text, Atlan replaces:

  • String literals with '<REDACTED:STRING>'.
  • Numeric literals with 1. Atlan substitutes a valid number rather than a text placeholder so that the redacted SQL remains parseable.

Atlan also removes comments from redacted SQL and reformats the statement, so stored text won't match the original layout.

Values Atlan preserves​

Some literals are safe to keep and can break the SQL if replaced, so redaction preserves them:

  • Date and time literals, such as '2026-01-01'::date or '2026-01-01 14:30:00'.
  • Month and weekday names, and date part keywords such as MONTH or QUARTER.
  • Currency codes, such as 'USD'.
  • Boolean words: TRUE, FALSE, YES, and NO.
  • Numbers in structural positions, such as the row count in a LIMIT clause, where the value controls the shape of the result rather than describing your data.

Everything else is treated as potentially sensitive and redacted.

Before and after​

A view definition extracted by the crawler:

select order_id, customer_email
from analytics.orders
where region = 'EMEA' and order_total > 500.75
  and created_at >= '2026-01-01' and currency = 'USD'
limit 100;

The same definition as Atlan stores it with Redact SQL Text enabled:

/* redacted: true, hash: <original_hash>, redacted_hash: <redacted_hash> */
SELECT
  order_id,
  customer_email
FROM analytics.orders
WHERE
  region = '<REDACTED:STRING>'
  AND order_total > 1
  AND created_at >= '2026-01-01'
  AND currency = 'USD'
LIMIT 100

Note the following:

  • 'EMEA' was replaced, while the date and the currency code were preserved.
  • 500.75 became 1. Redaction indicates that a number was present, not what it was.
  • LIMIT 100 was preserved, because the number controls the result size rather than describing your data.
  • The statement was reformatted.
  • Atlan adds a header comment recording that the text was redacted, along with fingerprints of the original and redacted statements. The fingerprint values differ for every statement.
  • analytics.orders, order_id, and customer_email are unchanged, which is why lineage still resolves.

What redaction preserves​

Redaction only changes literal values. Because everything Atlan uses to build relationships stays intact, enabling redaction doesn't change:

  • Table-level and column-level lineage.
  • Asset names, hierarchy, or any other extracted metadata.

Limitations​

Earlier miner runs stay unredacted. SQL text already stored in Atlan from earlier miner runs stays unredacted.

SQL that Atlan can't parse is stored unredacted. When parsing fails, Atlan stores the original statement under a redacted: false header instead of dropping it. Function and procedure bodies written in PL/pgSQL or another procedural language, and dynamic SQL built with EXECUTE, are the most common cases.

Redaction increases workflow runtime. Expect crawler and miner runs to take longer, and account for this if a workflow already runs close to its schedule interval.

Enabling redaction rewrites stored SQL. Because the stored text changes for every asset, the first run after you enable the option updates SQL text across the connection.

See also​