## Rule types and failed rows validations

URL: https://docs.atlan.com/product/capabilities/governance/data-quality/references/rule-types-and-failed-rows

> Reference of Data Quality Studio rule types, restrictions, and how failed rows validation works

Atlan's Data Quality Studio runs rule-based checks natively in Snowflake, Databricks, and BigQuery, returning the exact failed rows for each violation.

## How failed rows validation works

When a data quality rule fails, Atlan automatically generates SQL queries that help you identify and inspect the problematic rows in your source system.

**Query behavior:**
- Filters rows that violate the rule condition
- Available for rules with per-row conditions (including Min/Max). Not available for aggregate-only metrics (Average, Standard Deviation, Row Count, Freshness, and all Reconciliation rules). For Custom SQL, failed rows are available only when your SQL returns invalid rows.

**Engine support:**
- All rule types work with Snowflake, Databricks and BigQuery
- Column type restrictions vary by engine (see individual rule details below)

## Accessing failed rows

Failed rows SQL queries can be accessed from the Data quality tab in Atlan. Select any failed rule execution to view the generated SQL query to investigate the specific rows that violate the rule.

![Data quality failed rows SQL](https://docs.atlan.com/img/product-capabilities/governance/data-quality-studio/references/access-data-quality-failed-rows-sql.png)

## Available rule types

Rule types are organized by data quality dimension. Each rule type includes supported column types for Snowflake, Databricks and BigQuery, along with details about the failed rows SQL it generates.

- **Completeness**: Blank Count, Blank Percentage, Null Count, Null Percentage
- **Statistical**: Max Value, Min Value, Average, Standard Deviation
- **Uniqueness**: Unique Count, Duplicate Count
- **Validity**: Regex, String Length, Valid Values, Reference
- **Timeliness**: Freshness
- **Volume**: Row Count
- **Consistency**: Row Count Reconciliation, Average Reconciliation, Sum Reconciliation, Duplicate Count Reconciliation, Unique Count Reconciliation
- **Custom**: Custom SQL

### Completeness checks

Completeness rules identify missing or empty data in your columns.

<!-- vale off -->

### Snowflake

### Databricks

### BigQuery

<!-- vale on -->

### Statistical checks

Statistical rules validate numeric values by comparing them to minimum, maximum, average, and standard deviation thresholds.

<!-- vale off -->

### Snowflake

 X", "SELECT * FROM table WHERE column <= X"]}
/>

= X"]}
/>

### Databricks

 X", "SELECT * FROM table WHERE column <= X"]}
/>

= X"]}
/>

### BigQuery

 X", "SELECT * FROM table WHERE column <= X"]}
/>

= X"]}
/>

<!-- vale on -->

### Uniqueness checks

Uniqueness rules identify duplicate values and measure distinct value counts in columns.

<!-- vale off -->

### Snowflake

 1"
/>

### Databricks

 1"
/>

### BigQuery

 1"
/>

<!-- vale on -->

### Validity checks

Validity rules verify that data values conform to expected formats, patterns, and constraints.

<!-- vale off -->

### Snowflake

, ≥, <, ≤."
 enum={["VARCHAR", "STRING", "TEXT"]}
 example="SELECT * FROM table WHERE LENGTH(column) > X"
/>

### Databricks

, ≥, <, ≤."
 enum={["STRING"]}
 example="SELECT * FROM table WHERE LENGTH(column) > X"
/>

### BigQuery

, ≥, <, ≤."
 enum={["STRING"]}
 example="SELECT * FROM table WHERE LENGTH(column) > X"
/>

<!-- vale on -->

### Timeliness checks

Timeliness rules verify data freshness by checking when data was last updated.

<!-- vale off -->

### Snowflake

### Databricks

### BigQuery

<!-- vale on -->

### Volume checks

Volume rules measure the total number of rows in tables.

### Consistency checks

Consistency rules validate that data matches between source and target tables by comparing aggregate metrics. These rules are useful for data reconciliation scenarios where you need to confirm that data pipelines maintain integrity.

<!-- vale off -->

### Snowflake

### Databricks

### BigQuery

:::info Coming soon
Reconciliation rules for BigQuery Data Quality Studio are coming soon. Check back for updates on cross-table data validation capabilities.
:::

<!-- vale on -->

#### Reconciliation rule configuration

Reconciliation rules require configuring both a source and target for comparison:

- **Source table/column**: The primary table or column you want to validate
- **Target table/column**: The reference table or column to compare against
- **Allowed difference**: The maximum acceptable difference, configured as either:
 - **Percentage of source**: Difference as a percentage of the source value (e.g., 1% means the rule fails if the difference exceeds 1% of the source)
 - **Absolute value**: Exact numeric difference allowed between source and target

:::info Incremental processing
Reconciliation rules support row scope filtering for incremental processing. When enabled, the filter applies to both source and target tables, allowing you to compare recent data loads without scanning entire tables.
:::

### Custom checks

Custom rules let you define your own SQL-based metrics tailored to specific validation requirements.

#### Custom SQL return types

The two return type modes determine how your SQL query results are processed and whether failed rows are available.

) AS t` to evaluate thresholds and enables failed-rows SQL using your original query."
/>

---

## See also

- [Use AI-suggested rules](https://docs.atlan.com/llms/governance/data-quality/use-ai-suggested-rules/llms.txt): Let Atlan AI automatically suggest data quality rules based on your asset's metadata
- [Data Quality Studio concepts](https://docs.atlan.com/llms/governance/data-quality/data-quality-studio/llms.txt)
- [Set up Snowflake](https://docs.atlan.com/llms/governance/data-quality/set-up-snowflake/llms.txt)
- [Set up Databricks](https://docs.atlan.com/llms/governance/data-quality/set-up-databricks/llms.txt)
- [Set up BigQuery](https://docs.atlan.com/llms/governance/data-quality/set-up-bigquery/llms.txt)

---