Skip to main content

Rule types and failed rows validations

Data Quality Studio provides rule types to validate data quality in Snowflake and Databricks. Each rule type targets specific data quality dimensions and generates a SQL query which provides failed rows when violations occur.

How failed rows validation works​

When a data quality rule fails, Atlan automatically generates SQL queries that help you identify and inspect the problematic rows in your source system.

Query behavior:

  • Filters rows that violate the rule condition
  • Available for rules with per-row conditions (including Min/Max). Not available for aggregate-only metrics (Average, Standard Deviation, Row Count, Freshness, Custom SQL)

Engine support:

  • All rule types work with both Snowflake and Databricks
  • Column type restrictions vary by engine (see individual rule details below)

Accessing failed rows​

Failed rows SQL queries can be accessed from the Data quality tab in Atlan. Select any failed rule execution to view the generated SQL query for investigating the specific rows that violate the rule.

Data quality failed rows SQL

Available rule types​

Rule types are organized by data quality dimension. Each rule type includes supported column types for Snowflake and Databricks, along with details about the failed rows SQL it generates.

Completeness checks​

Completeness rules identify missing or empty data in your columns.

Blank CountCompleteness
Optional

Counts blank values in a column.

Allowed values:
VARCHARSTRINGTEXT
Example:SELECT * FROM table WHERE column = ''

Blank PercentageCompleteness
Optional

Calculates the percentage of blank values in a column.

Allowed values:
VARCHARSTRINGTEXT
Example:SELECT * FROM table WHERE column = ''

Null CountCompleteness
Optional

Counts NULL values in a column.

Allowed values:
DATEFLOATNUMBERDECIMALDECNUMERICINTINTEGERBIGINTSMALLINTFLOAT4FLOAT8DOUBLEDOUBLE PRECISIONREALTIMESTAMP_LTZTIMESTAMP_NTZDATETIMETIMESTAMPLTZTIMESTAMPNTZTIMESTAMP_TZVARCHARSTRINGTEXT
Example:SELECT * FROM table WHERE column IS NULL

Null PercentageCompleteness
Optional

Calculates the percentage of NULL values in a column.

Allowed values:
DATEFLOATNUMBERDECIMALDECNUMERICINTINTEGERBIGINTSMALLINTFLOAT4FLOAT8DOUBLEDOUBLE PRECISIONREALTIMESTAMP_LTZTIMESTAMP_NTZDATETIMETIMESTAMPLTZTIMESTAMPNTZTIMESTAMP_TZVARCHARSTRINGTEXT
Example:SELECT * FROM table WHERE column IS NULL

Statistical checks​

Statistical rules validate numeric values by comparing them to minimum, maximum, average, and standard deviation thresholds.

Max ValueStatistical
Optional

Computes the maximum value and validates it against a threshold. Provides failed rows that violate the threshold (for example, when using ≥ X, returns rows less than X).

Allowed values:
FLOATNUMBERDECIMALDECNUMERICINTINTEGERBIGINTSMALLINTFLOAT4FLOAT8DOUBLEDOUBLE PRECISIONREAL
Examples:
  • SELECT * FROM table WHERE column > X
  • SELECT * FROM table WHERE column <= X

Min ValueStatistical
Optional

Computes the minimum value and validates it against a threshold. Provides failed rows that violate the threshold (for example, when using ≥ X, returns rows less than X).

Allowed values:
FLOATNUMBERDECIMALDECNUMERICINTINTEGERBIGINTSMALLINTFLOAT4FLOAT8DOUBLEDOUBLE PRECISIONREAL
Examples:
  • SELECT * FROM table WHERE column < X
  • SELECT * FROM table WHERE column >= X

AverageStatistical
Metric-only

Calculates the average value of numeric columns. Failed rows sql query is not available.

Allowed values:
FLOATNUMBERDECIMALDECNUMERICINTINTEGERBIGINTSMALLINTFLOAT4FLOAT8DOUBLEDOUBLE PRECISIONREAL

Standard DeviationStatistical
Metric-only

Calculates standard deviation for numeric columns. Failed rows sql query is not available.

Allowed values:
FLOATNUMBERDECIMALDECNUMERICINTINTEGERBIGINTSMALLINTFLOAT4FLOAT8DOUBLEDOUBLE PRECISIONREAL

Uniqueness checks​

Uniqueness rules identify duplicate values and measure distinct value counts in columns.

Unique CountUniqueness
Optional

Counts the number of unique non-NULL values in the selected column.

Allowed values:
DATEFLOATNUMBERDECIMALDECNUMERICINTINTEGERBIGINTSMALLINTFLOAT4FLOAT8DOUBLEDOUBLE PRECISIONREALTIMESTAMP_LTZTIMESTAMP_NTZDATETIMETIMESTAMPLTZTIMESTAMPNTZTIMESTAMP_TZVARCHARSTRINGTEXT
Example:SELECT DISTINCT column FROM table

Duplicate CountUniqueness
Optional

Counts distinct values that appear more than once.

Allowed values:
DATEFLOATNUMBERDECIMALDECNUMERICINTINTEGERBIGINTSMALLINTFLOAT4FLOAT8DOUBLEDOUBLE PRECISIONREALTIMESTAMP_LTZTIMESTAMP_NTZDATETIMETIMESTAMPLTZTIMESTAMPNTZTIMESTAMP_TZVARCHARSTRINGTEXT
Example:SELECT column, COUNT(*) FROM table GROUP BY column HAVING COUNT(*) > 1

Validity checks​

Validity rules verify that data values conform to expected formats, patterns, and constraints.

RegexValidity
Optional

Validates column values using REGEXP function. Returns rows that don't satisfy the regex condition.

Allowed values:
VARCHARSTRINGTEXT
Example:SELECT * FROM table WHERE NOT REGEXP(column, 'pattern')

String LengthValidity
Optional

Validates string length using comparison operators. Returns rows where string length violates the threshold. Supported operators: between, =, >, ≥, <, ≤.

Allowed values:
VARCHARSTRINGTEXT
Example:SELECT * FROM table WHERE LENGTH(column) > X

Valid ValuesValidity
Optional

Validates values using allowed or disallowed lists. Returns rows with values outside the allowed list or inside the disallowed list.

Allowed values:
VARCHARSTRINGTEXT
Example:SELECT * FROM table WHERE column NOT IN ('value1', 'value2', 'value3')

Timeliness checks​

Timeliness rules verify data freshness by checking when data was last updated.

FreshnessTimeliness
Metric-only

Validates data recency using date or timestamp columns. Failed rows sql query is not available.

Allowed values:
DATETIMESTAMP_LTZTIMESTAMP_TZTIMESTAMPLTZTIMESTAMPTZ

Volume checks​

Volume rules measure the total number of rows in tables.

Row CountVolume
Metric-only

Counts total rows in a table. Operates at table level (no column selection required). Failed rows sql query not available.

Custom checks​

Custom rules let you define your own SQL-based metrics tailored to specific validation requirements.

Custom SQLCustom
Metric-only

Executes custom SQL queries that return a single numeric result. Failed rows sql query is not available.

See also​