Anomaly detection results

This document describes the data model and result fields for anomaly detection rules in Data Quality Studio, including how results flow from Snowflake to Atlan and what each field represents.

Rule attributes

When anomaly detection is enabled, two DataQualityRule entities are created with the following templates:

Rule	Template name	Dimension	Description
Anomaly Detection - Row Count	ANOMALY_DETECTION_ROW_COUNT	Volume	Monitors row count using Snowflake's native anomaly detection
Anomaly Detection - Freshness	ANOMALY_DETECTION_FRESHNESS	Timeliness	Monitors data freshness using Snowflake's native anomaly detection

Result fields

Each anomaly detection result includes the standard DQ rule result fields plus additional ML-specific details.

Standard fields

Field	Attribute	Type	Description
Latest result	dqRuleLatestResult	String	PASS or FAIL, indicating whether the actual value is within the expected range
Latest metric value	dqRuleLatestMetricValue	String	The actual measured value (for example, the row count or freshness value)
Result computed at	dqRuleLatestResultComputedAt	Epoch (ms)	Timestamp when the result was evaluated
Metric computed at	dqRuleLatestMetricValueComputedAt	Epoch (ms)	Timestamp when the metric was measured by the DMF

Anomaly detection fields

These fields are stored in the dqRuleLatestResultDetails attribute as a JSON string:

Field	Key	Type	Description
Forecast	forecast	Number	The ML-predicted expected value based on historical patterns
Upper bound	upper_bound	Number	The upper limit of the normal range
Lower bound	lower_bound	Number	The lower limit of the normal range
Is anomaly	is_anomaly	Boolean	true if the actual value falls outside the expected range; false otherwise

Example dqRuleLatestResultDetails payload:

{
  "forecast": 1500.0,
  "upper_bound": 1650.0,
  "lower_bound": 1350.0,
  "is_anomaly": false
}

Status values

The dqRuleADStatus attribute on the DataQualityRule entity tracks the anomaly detection lifecycle:

Status	Description	Transition trigger
TRAINING	ML model is collecting data and learning patterns	Set when the rule is created (toggle turned on)
ACTIVE	Model is producing predictions; results are flowing	Set when the first AD results are published to Atlas
ERROR	Sync to Snowflake failed	Set when the DQ event processor fails to sync the rule

Data flow

Anomaly detection results follow this pipeline from Snowflake to Atlan:

Snowflake DMF execution (with ANOMALY_DETECTION = TRUE)
    ↓
SNOWFLAKE.LOCAL.DATA_QUALITY_MONITORING_ANOMALY_DETECTION_STATUS view
    ↓
Scheduled task MERGEs into DQ_RULE_RESULT (with RESULT_DETAILS column)
    ↓
fetch_dq_results reads DQ_RULE_RESULT (including RESULT_DETAILS)
    ↓
Published to Kafka DQ_METADATA topic
    ↓
Numaflow UDF renders Jinja2 template → Atlas bulk update
    ↓
dqRuleLatestResult, dqRuleLatestMetricValue, dqRuleLatestResultDetails, dqRuleADStatus

Snowflake storage

In the customer's Snowflake warehouse, anomaly detection results are stored in DQ_RULE_RESULT with the RESULT_DETAILS column (type VARIANT) containing the ML output:

OBJECT_CONSTRUCT(
    'forecast', forecast_value,
    'upper_bound', upper_bound_value,
    'lower_bound', lower_bound_value,
    'is_anomaly', is_anomaly_flag
)::VARIANT

Atlas storage

The Numaflow pipeline publishes the following attributes to the DataQualityRule entity in Atlas:

• dqRuleLatestResult: PASS or FAIL
• dqRuleLatestMetricValue: the actual measured value
• dqRuleLatestResultDetails: the JSON string with forecast, bounds, and anomaly flag
• dqRuleADStatus: set to ACTIVE when results with result_details are present

See also

• What's anomaly detection: Understand how anomaly detection works
• Enable anomaly detection: Step-by-step guide to enable anomaly detection
• Rule types and failed rows validations: Reference for all DQ rule types