Monitors

Monitor your data for schema changes, freshness, anomalies, and column-level statistics

Monitors in Sling observe your data assets without moving data. They track schema drift, data freshness, row counts, column-level statistics, and anomalies β€” giving you continuous visibility into the health and quality of your database tables and views.

monitor-charts

What Can Monitors Do?

  • Schema change detection β€” Track when tables, columns, or types are added, dropped, or altered

  • Data freshness β€” Alert when data falls behind its expected update schedule

  • Row count tracking β€” Monitor table sizes and detect unexpected growth or shrinkage

  • Column statistics β€” Collect min/max/mean, percentiles, null counts, cardinality, and more

  • Value validation β€” Enforce regex patterns, accepted values, and rejected values on columns

  • Anomaly detection β€” Automatically detect spikes, drops, and pattern changes using z-score analysis

Quick Start

Create a YAML file that specifies which connection to monitor and what to track:

On the Sling Platform, save this file in the monitors/ directory of your project and schedule it as a job.

circle-check

Configuration Overview

A monitor configuration has five top-level keys:

Key
Required
Description

connection

Yes

The connection to monitor (must match a connection in env.yaml)

defaults

No

Default settings applied to all objects. See Structure.

objects

Yes

Map of object names or patterns to their monitoring configuration. See Structure.

schemata

No

Schema change detection settings. See Schema Changes.

For the complete YAML reference, see Structure.

Running Monitors

Monitor files are stored in the monitors/ directory of your project on the Sling Platform. Once created, you can schedule them as jobs with a cron expression for automated execution. Monitor runs process objects in parallel (default 2 threads) and retry transient errors up to 3 times.

Supported Connections

Databases

Database
Table Size
Percentiles
Freshness

PostgreSQL

Yes

Yes

Yes

MySQL / MariaDB

Yes

β€”

Yes

Snowflake

Yes

Yes

Yes

BigQuery

Yes

Yes (approx)

Yes

ClickHouse

Yes

Yes

Yes

DuckDB

Yes

Yes

Yes

SQL Server

Yes

Yes

Yes

Oracle

Yes

Yes

Yes

Redshift

Yes

Yes

Yes

Databricks

Yes

Yes

Yes

Trino / Athena

Yes

Yes

Yes

SQLite / D1

β€”

β€”

Yes

Notifications

circle-info

Notification settings are a Sling Platform feature, configured per monitor job.

monitor-notification

Each monitor job supports four notification triggers:

Setting
Fires When

On Schema Change

Tables, columns, or types are added, dropped, or altered

On Freshness

Data exceeds its freshness threshold

On Anomaly

Metric values deviate significantly from historical baseline

On Failure

Monitor execution fails

Notifications can be sent via email, Slack, Discord, or Microsoft Teams.

Learn More

Detailed documentation for each aspect of monitors:

Last updated

Was this helpful?