AWS SES

Connect & Ingest email transaction data from AWS SES

Amazon Simple Email Service (SES) is a cloud-based email sending service designed to help businesses send marketing, notification, and transactional emails. The Sling AWS SES connector extracts email transaction data including delivery status, opens, clicks, bounces, and complaints through the Message Insights export API.

CLI Pro Required: APIs require a CLI Pro token or Platform Plan.

Setup

The following credentials and inputs are accepted:

Secrets:

• access_key_id (optional) -> Your AWS Access Key ID

• secret_access_key (optional) -> Your AWS Secret Access Key

• session_token (optional) -> Your AWS Session Token (for temporary credentials)

• profile (optional) -> AWS profile name from your credentials file

Inputs:

• region (required) -> AWS region where SES is configured (e.g., us-east-1, eu-west-1)

• anchor_date (optional) -> Starting date for first sync (default: 30 days ago). Format: YYYY-MM-DD

• range_interval (optional) -> Time interval per export chunk (default: 1d). Format: integer + unit suffix. Supported units: s (second), m (minute), h (hour), d (day), mo (month). Examples: 1d, 6h, 30m.

• lookback_interval (optional) -> How far back to look from the last sync point (default: 0d). Useful for catching late-arriving data. Format: same as range_interval. Examples: 1h, 30m.

High-Volume Accounts: AWS SES exports are limited to 10,000 records per request. The connector will fail with an error if it hits this limit to prevent data loss. If you send more than 10,000 emails per day, set range_interval: 1h for hourly exports, range_interval: 30m for 30-minute chunks, or lower as needed.

Authentication: You can authenticate using either explicit credentials (access_key_id + secret_access_key) or an AWS profile. If using IAM roles or instance profiles, the connector will automatically use the environment's default credentials.

IAM Permissions

Your AWS credentials need the following IAM permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ses:CreateExportJob",
        "ses:GetExportJob"
      ],
      "Resource": "*"
    }
  ]
}

Additionally, SES exports the data to an S3 bucket, so access to read the exported data from S3 is handled through a pre-signed URL that SES provides.

Getting Your AWS Credentials

1. Log in to the AWS Console

2. Go to IAM > Users > Select your user

3. Go to Security credentials tab

4. Click Create access key

5. Copy the Access key ID and Secret access key

Important: Store your credentials securely. The secret access key is only shown once when created.

Using sling conns

sling conns set AWS_SES type=api spec=aws_ses inputs='{ region: us-east-1 }' secrets='{ access_key_id: AKIAXXXXXXXX, secret_access_key: xxxxxxxx }'

Using an AWS profile:

sling conns set AWS_SES type=api spec=aws_ses inputs='{ region: us-east-1 }' secrets='{ profile: my-profile }'

Environment Variable

export AWS_SES='{ type: api, spec: aws_ses, inputs: { region: "us-east-1" }, secrets: { access_key_id: "AKIAXXXXXXXX", secret_access_key: "xxxxxxxx" } }'

Sling Env File YAML

See here to learn more about the sling env.yaml file.

connections:
  AWS_SES:
    type: api
    spec: aws_ses
    inputs:
      region: "us-east-1"
    secrets:
      access_key_id: "AKIAXXXXXXXX"
      secret_access_key: "xxxxxxxx"

Using an AWS profile:

connections:
  AWS_SES:
    type: api
    spec: aws_ses
    inputs:
      region: "us-east-1"
    secrets:
      profile: "my-profile"

Replication

Here's an example replication configuration to sync AWS SES email data to a PostgreSQL database:

source: AWS_SES
target: MY_POSTGRES

defaults:
  mode: incremental
  object: ses.{stream_table}

streams:
  ses_emails:

Full refresh example:

source: AWS_SES
target: MY_POSTGRES

defaults:
  mode: full-refresh
  object: ses.{stream_table}

streams:
  ses_emails:

Endpoints

Endpoint	Description	Incremental
ses_emails	Email transaction data including delivery status, opens, clicks, bounces, and complaints	Yes

To discover available endpoints:

sling conns discover AWS_SES

How It Works

The AWS SES connector uses the Message Insights export feature:

1. Export Job Creation: Creates an export job via the SES API for the specified date range

2. Job Polling: Polls the job status until the export is complete (with automatic retries)

3. Data Download: Once complete, SES provides a pre-signed S3 URL to download the exported data

4. Data Processing: The connector downloads and processes the JSON data

5. Sync Tracking: Tracks the last synced date for incremental sync on subsequent runs

Export Time: Depending on the volume of email data, export jobs may take a few minutes to complete. The connector automatically handles the waiting and retries.

Incremental Sync

The AWS SES connector supports incremental sync:

• First run: Fetches email data from anchor_date (default: 30 days ago) forward by range_interval (default: 1d)

• Subsequent runs: Fetches email data from the last synced date forward by range_interval

• Lookback: Use lookback_interval to re-fetch data from before the last sync point, useful for catching late-arriving events

The primary key for deduplication is messageid.

Configuring for High-Volume Accounts

If you send a large volume of emails and hit the 10,000 record limit, adjust the chunk size:

connections:
  AWS_SES:
    type: api
    spec: aws_ses
    inputs:
      region: "us-east-1"
      range_interval: 1h  # Hourly chunks for >10K emails/day
    secrets:
      profile: "my-profile"

For extremely high-volume accounts (>10K emails/hour), use sub-hour chunks:

inputs:
  region: "us-east-1"
  range_interval: 30m  # 30-minute chunks

If you are facing issues connecting, please reach out to us at [email protected], on discord or open a Github Issue here.