# Anomaly Detection

Sling automatically detects anomalies in your monitored metrics using **z-score analysis** over historical data. No extra configuration is needed — just enable the metrics you care about and Sling will alert you when values deviate significantly from their historical baseline.

## How It Works

After collecting sufficient historical data, Sling calculates a **baseline** (mean and standard deviation) from recent metric values. Each new value is compared against this baseline by computing a z-score:

```
z-score = |current_value - mean| / standard_deviation
```

When the z-score exceeds the configured threshold (default: 3.0), an anomaly event is fired — either a **spike** (value significantly higher than expected) or a **drop** (value significantly lower than expected).

## Requirements

Anomaly detection activates automatically once sufficient history is available:

| Parameter            | Default | Description                                      |
| -------------------- | ------- | ------------------------------------------------ |
| Minimum data points  | 7       | Number of historical measurements needed         |
| Minimum history days | 7       | Days of data collection before detection begins  |
| Lookback window      | 30 days | How far back to look when computing the baseline |

All parameters are configurable via the `anomaly_detection` block.

{% hint style="info" %}
Anomaly detection begins automatically after collecting at least 7 data points over at least 7 days. Until then, metrics are tracked but no anomaly events are fired.
{% endhint %}

## What Gets Checked

**Object-level:** Row count changes are checked automatically when `row_count: true` is set.

**Column-level:** Any column metric you enable is tracked for anomalies over time. This includes `count`, `null_count`, `count_distinct`, `min_max_mean`, `percentile`, `size`, and others.

**Validation rules:** Violations from `accepted_values`, `rejected_values`, `regex_match`, and `regex_not_match` are also reported as anomaly events.

```yaml
objects:
  public.orders:
    row_count: true              # anomaly detection on row count
    columns:
      revenue:
        min_max_mean: true       # detect revenue spikes/drops
        null_count: true         # detect null count changes
      status:
        count_distinct: true     # detect cardinality shifts
```

## Configuration

Use `anomaly_detection` in `defaults` or per object to tune detection sensitivity:

| Key                  | Type  | Default | Description                                         |
| -------------------- | ----- | ------- | --------------------------------------------------- |
| `z_score_threshold`  | float | `3.0`   | Z-score threshold — lower values are more sensitive |
| `min_history_points` | int   | `7`     | Minimum data points before detection activates      |
| `min_history_days`   | int   | `7`     | Minimum days of history required                    |
| `history_days`       | int   | `30`    | Lookback window for baseline calculation            |

Per-object settings override individual fields from defaults; unset fields inherit from defaults.

## Event Types

| Event                        | Description                                 |
| ---------------------------- | ------------------------------------------- |
| `anomaly_spike`              | Value is significantly higher than expected |
| `anomaly_drop`               | Value is significantly lower than expected  |
| `anomaly_pattern_change`     | Overall pattern shift detected              |
| `anomaly_validation_failure` | Validation rule violations detected         |

## Severity Levels

Anomaly severity is determined by the z-score magnitude:

| Severity | Z-Score Range             |
| -------- | ------------------------- |
| Low      | ≥ 2.0                     |
| Medium   | ≥ 3.0 (default threshold) |
| High     | ≥ 4.0                     |
| Critical | ≥ 5.0                     |

## Examples

### Default Detection

Enable metrics and let Sling handle anomaly detection with default settings:

```yaml
connection: MY_POSTGRES

defaults:
  row_count: true

objects:
  public.orders:
    columns:
      revenue:
        min_max_mean: true
      quantity:
        null_count: true
  public.users:
    columns:
      email:
        count_distinct: true
```

### Custom Thresholds

Set a lower z-score threshold for sensitive tables:

```yaml
connection: MY_POSTGRES

defaults:
  row_count: true
  anomaly_detection:
    z_score_threshold: 2.5       # more sensitive globally
    min_history_points: 10       # require more history

objects:
  public.orders:
    anomaly_detection:
      z_score_threshold: 2.0     # even more sensitive for orders
    columns:
      revenue:
        min_max_mean: true

  public.users:
    row_count: true              # inherits defaults (2.5 / 10)
```

### Longer History Window

For data with seasonal patterns, use a longer lookback window:

```yaml
connection: MY_POSTGRES

defaults:
  anomaly_detection:
    history_days: 90             # 3-month window for seasonal data
    min_history_points: 14       # require 2 weeks of data

objects:
  public.monthly_sales:
    row_count: true
    columns:
      total_revenue:
        min_max_mean: true
```

## Notifications

Enable **On Anomaly** in your monitor job's notification settings to receive alerts when anomalies are detected. Notifications can be sent via email, Slack, Discord, or Microsoft Teams.

{% hint style="info" %}
Notification settings are configured per monitor job on the Sling Platform. See the [Platform documentation](https://docs.slingdata.io/sling-platform/platform) for details on setting up notification channels.
{% endhint %}