Cloudflare Analytics Engine

Connect & Ingest data from Cloudflare Analytics Engine

Cloudflare Analytics Engine is a time-series analytics database that allows you to store and query custom metrics from your Cloudflare Workers. The Sling Cloudflare Analytics Engine connector extracts data via the SQL API, automatically discovering all tables (datasets) in your account.

CLI Pro Required: APIs require a CLI Pro token or Platform Plan.

Setup

The following credentials and inputs are accepted:

Secrets:

• api_token (required) -> Your Cloudflare API Token with Analytics Engine read permissions

• account_id (required) -> Your Cloudflare Account ID

Inputs:

• anchor_date (optional) -> The starting date for historical data extraction (default: 90 days ago). Format: YYYY-MM-DD

Getting Your Credentials

Account ID

1. Log in to your Cloudflare Dashboard

2. Select any domain in your account

3. Your Account ID is displayed in the right sidebar under "API"

API Token

1. Go to Cloudflare API Tokens

2. Click Create Token

3. Select Create Custom Token

4. Configure the token:

   • Token name: Sling Analytics Engine (or your preferred name)

   • Permissions: Account > Account Analytics > Read

   • Account Resources: Include your account

5. Click Continue to summary and then Create Token

6. Copy the token immediately (it won't be shown again)

Important: Make sure your API token has the Account Analytics: Read permission. Standard API keys won't work—you need a scoped API token.

Using sling conns

sling conns set CF_ANALYTICS type=api spec=cloudflare_analytics_engine secrets='{ api_token: your_token_here, account_id: your_account_id }'

Environment Variable

export CF_ANALYTICS='{ type: api, spec: cloudflare_analytics_engine, secrets: { api_token: "your_token_here", account_id: "your_account_id" } }'

Sling Env File YAML

See here to learn more about the sling env.yaml file.

connections:
  CF_ANALYTICS:
    type: api
    spec: cloudflare_analytics_engine
    secrets:
      api_token: "your_token_here"
      account_id: "your_account_id"

With custom anchor date:

connections:
  CF_ANALYTICS:
    type: api
    spec: cloudflare_analytics_engine
    secrets:
      api_token: "your_token_here"
      account_id: "your_account_id"
    inputs:
      anchor_date: "2024-01-01"

Dynamic Endpoints

The Cloudflare Analytics Engine connector uses dynamic endpoints to automatically discover all tables (datasets) in your account. When you run sling conns discover, the connector:

1. Queries SHOW TABLES via the SQL API

2. Creates an endpoint for each discovered table

3. Names each endpoint using snake_case (e.g., my_dataset for a table named "MyDataset")

To see available endpoints:

sling conns discover CF_ANALYTICS

Replication

Here's an example replication configuration to sync Analytics Engine data to a PostgreSQL database:

source: CF_ANALYTICS
target: MY_POSTGRES

defaults:
  mode: incremental
  object: analytics.{stream_table}

streams:
  # sync all discovered tables
  '*':

Full refresh example:

source: CF_ANALYTICS
target: MY_POSTGRES

defaults:
  mode: full-refresh
  object: analytics.{stream_table}

streams:
  # sync specific tables
  my_metrics:
  page_views:
  api_requests:

Incremental Sync

The connector uses time-based incremental sync with the timestamp field:

• First run: Fetches all records from anchor_date (default: 90 days ago) to present

• Subsequent runs: Only fetches records with timestamp >= last synced timestamp

The connector automatically:

• Tracks the maximum timestamp value from each run

• Uses this as the starting point for the next run

• Paginates through large datasets (10,000 records per page)

Analytics Engine data has a 90-day retention by default. Set anchor_date appropriately based on your retention settings.

Data Schema

Analytics Engine tables have a flexible schema. Common columns include:

Column	Description
timestamp	Event timestamp (used for incremental sync)
index1	First index dimension
blob1...blob20	String blob columns
double1...double20	Numeric double columns

The exact columns depend on how you've configured your Analytics Engine datasets.

Rate Limiting

The connector is configured with conservative rate limiting:

• Rate: 5 requests per second

• Concurrency: 3 parallel requests

• Retry: Exponential backoff on 429 (rate limit) responses

Common Use Cases

Sync All Analytics Data

source: CF_ANALYTICS
target: MY_SNOWFLAKE

defaults:
  mode: incremental
  object: cloudflare.{stream_table}

streams:
  '*':

Sync Specific Datasets

source: CF_ANALYTICS
target: MY_POSTGRES

defaults:
  mode: incremental
  object: metrics.{stream_table}

streams:
  page_views:
  api_latency:
  error_rates:

Historical Backfill

source: CF_ANALYTICS
target: MY_POSTGRES

defaults:
  mode: full-refresh
  object: analytics.{stream_table}

env:
  ANCHOR_DATE: "2024-06-01"

streams:
  '*':

If you are facing issues connecting, please reach out to us at [email protected], on discord or open a Github Issue here.