Skip to main content

Manage data quality rules

Create data quality rules

Data quality rules can be created using three different creator methods depending on the type of rule you want to create:

  1. Column level rules: For rules that apply to specific columns (for example, Freshness, Null Count)
  2. Table level rules: For rules that apply to entire tables (for example, Row Count)
  3. Custom SQL rules: For Custom SQL rule only

Column level rules

Column level rules are used for data quality checks that apply to specific columns within a asset.

Coming soon

Rule types with conditions

These rule types (String Length, Regex, Valid Values) make data validation faster and easier by providing common checks that previously required custom SQL or complex setup. They support adding rule conditions along with thresholds.

Coming soon

Reconciliation rules

Reconciliation rules allow you to compare metrics between a base table/column and a target table/column. These rules support reconciliation conditions that specify the target table and column for comparison.

Coming soon

Table Level rules

Table level rules are used for data quality checks that apply to entire table.

Coming soon

Custom SQL rule

Custom SQL rule allow you to define data quality check using custom SQL queries.

Coming soon

Update data quality rules

To update an existing data quality rule, you only need to provide the qualified name and the Atlan Client. All other parameters are optional and will only be updated if provided.

Coming soon

Retrieve data quality rules

To retrieve data quality rules, you can use fluent search to retireve data quality rules.

Coming soon

Delete data quality rules

To delete data quality rules, you can use the standard asset deletion method.

Coming soon

Set row scope filter column

To configure incremental data quality monitoring, use the set_dq_row_scope_filter_column method. This enables rules to scan only the most recent day's data instead of entire datasets, making monitoring faster and more efficient for batch-based ingestions.

Coming soon

Schedule data quality rules

To add a schedule for data quality rules on an asset, you can use the add_dq_rule_schedule method. This method allows you to set up the schedule for data quality rule execution.

Coming soon
Was this page helpful?