Anomaly Detection

Automatic z-score anomaly detection for data quality monitoring

Sling automatically detects anomalies in your monitored metrics using z-score analysis over historical data. No extra configuration is needed — just enable the metrics you care about and Sling will alert you when values deviate significantly from their historical baseline.

How It Works

After collecting sufficient historical data, Sling calculates a baseline (mean and standard deviation) from recent metric values. Each new value is compared against this baseline by computing a z-score:

z-score = |current_value - mean| / standard_deviation

When the z-score exceeds the configured threshold (default: 3.0), an anomaly event is fired — either a spike (value significantly higher than expected) or a drop (value significantly lower than expected).

Requirements

Anomaly detection activates automatically once sufficient history is available:

Parameter	Default	Description
Minimum data points	7	Number of historical measurements needed
Minimum history days	7	Days of data collection before detection begins
Lookback window	30 days	How far back to look when computing the baseline

All parameters are configurable via the anomaly_detection block.

Anomaly detection begins automatically after collecting at least 7 data points over at least 7 days. Until then, metrics are tracked but no anomaly events are fired.

What Gets Checked

Object-level: Row count changes are checked automatically when row_count: true is set.

Column-level: Any column metric you enable is tracked for anomalies over time. This includes count, null_count, count_distinct, min_max_mean, percentile, size, and others.

Validation rules: Violations from accepted_values, rejected_values, regex_match, and regex_not_match are also reported as anomaly events.

objects:
  public.orders:
    row_count: true              # anomaly detection on row count
    columns:
      revenue:
        min_max_mean: true       # detect revenue spikes/drops
        null_count: true         # detect null count changes
      status:
        count_distinct: true     # detect cardinality shifts

Configuration

Use anomaly_detection in defaults or per object to tune detection sensitivity:

Key	Type	Default	Description
z_score_threshold	float	3.0	Z-score threshold — lower values are more sensitive
min_history_points	int	7	Minimum data points before detection activates
min_history_days	int	7	Minimum days of history required
history_days	int	30	Lookback window for baseline calculation

Per-object settings override individual fields from defaults; unset fields inherit from defaults.

Event Types

Event	Description
anomaly_spike	Value is significantly higher than expected
anomaly_drop	Value is significantly lower than expected
anomaly_pattern_change	Overall pattern shift detected
anomaly_validation_failure	Validation rule violations detected

Severity Levels

Anomaly severity is determined by the z-score magnitude:

Severity	Z-Score Range
Low	≥ 2.0
Medium	≥ 3.0 (default threshold)
High	≥ 4.0
Critical	≥ 5.0

Examples

Default Detection

Enable metrics and let Sling handle anomaly detection with default settings:

connection: MY_POSTGRES

defaults:
  row_count: true

objects:
  public.orders:
    columns:
      revenue:
        min_max_mean: true
      quantity:
        null_count: true
  public.users:
    columns:
      email:
        count_distinct: true

Custom Thresholds

Set a lower z-score threshold for sensitive tables:

connection: MY_POSTGRES

defaults:
  row_count: true
  anomaly_detection:
    z_score_threshold: 2.5       # more sensitive globally
    min_history_points: 10       # require more history

objects:
  public.orders:
    anomaly_detection:
      z_score_threshold: 2.0     # even more sensitive for orders
    columns:
      revenue:
        min_max_mean: true

  public.users:
    row_count: true              # inherits defaults (2.5 / 10)

Longer History Window

For data with seasonal patterns, use a longer lookback window:

connection: MY_POSTGRES

defaults:
  anomaly_detection:
    history_days: 90             # 3-month window for seasonal data
    min_history_points: 14       # require 2 weeks of data

objects:
  public.monthly_sales:
    row_count: true
    columns:
      total_revenue:
        min_max_mean: true

Notifications

Enable On Anomaly in your monitor job's notification settings to receive alerts when anomalies are detected. Notifications can be sent via email, Slack, Discord, or Microsoft Teams.

Notification settings are configured per monitor job on the Sling Platform. See the Platform documentation for details on setting up notification channels.