Sling
Slingdata.ioBlogGithubHelp!
  • Introduction
  • Sling CLI
    • Installation
    • Environment
    • Running Sling
    • Global Variables
    • CLI Pro
  • Sling Platform
    • Sling Platform
      • Architecture
      • Agents
      • Connections
      • Editor
      • API
      • Deploy from CLI
  • Concepts
    • Replications
      • Structure
      • Modes
      • Source Options
      • Target Options
      • Columns
      • Transforms
      • Runtime Variables
      • Tags & Wildcards
    • Hooks / Steps
      • Check
      • Command
      • Copy
      • Delete
      • Group
      • Http
      • Inspect
      • List
      • Log
      • Query
      • Replication
      • Store
      • Read
      • Write
    • Pipelines
    • Data Quality
      • Constraints
  • Examples
    • File to Database
      • Custom SQL
      • Incremental
    • Database to Database
      • Custom SQL
      • Incremental
      • Backfill
    • Database to File
      • Incremental
    • Sling + Python 🚀
  • Connections
    • Database Connections
      • Athena
      • BigTable
      • BigQuery
      • Cloudflare D1
      • Clickhouse
      • Databricks
      • DuckDB
      • DuckLake
      • Iceberg
      • MotherDuck
      • MariaDB
      • MongoDB
      • Elasticsearch
      • MySQL
      • Oracle
      • Postgres
      • Prometheus
      • Proton
      • Redshift
      • S3 Tables
      • StarRocks
      • SQLite
      • SQL Server
      • Snowflake
      • Trino
    • Storage Connections
      • AWS S3
      • Azure Storage
      • Backblaze B2
      • Cloudflare R2
      • DigitalOcean Spaces
      • FTP
      • Google Drive
      • Google Storage
      • Local Storage
      • Min.IO
      • SFTP
      • Wasabi
Powered by GitBook
On this page
  • Setup
  • Using sling conns
  • Environment Variable
  • Sling Env File YAML
  1. Connections
  2. Storage Connections

AWS S3

Connect & Ingest data from / into a AWS S3 Bucket

Setup

The following credentials keys are accepted:

  • bucket (required)

  • use_environment (optional) -> whether to use the AWS environment variables to access. Accepts true or false.

  • profile (optional) (or provide environment variable AWS_PROFILE)

  • access_key_id (optional) (or provide environment variable AWS_ACCESS_KEY_ID)

  • secret_access_key (optional) (or provide environment variable AWS_SECRET_ACCESS_KEY)

  • session_token (optional) (or provide environment variable AWS_SESSION_TOKEN)

  • region (optional) (or provide environment variable AWS_REGION)

  • endpoint (optional) (or provide environment variable AWS_ENDPOINT)

  • role_arn (optional) (or provide environment variable AWS_ROLE_ARN)

  • encryption_algorithm (optional since v1.2.15) The server-side encryption algorithm to use. Accepts AES256, aws:kms or aws:kms:arn.

  • encryption_kms_key (optional since v1.2.15) If encryption_algorithm is aws:kms or aws:kms:arn, this is the KMS key ID or ARN to use for encryption.

Using sling conns

Here are examples of setting a connection named AWS_S3. We must provide the type=s3 property:

$ sling conns set AWS_S3 type=s3 bucket=sling-bucket profile=my-profile

$ sling conns set AWS_S3 type=s3 bucket=sling-bucket access_key_id=ACCESS_KEY_ID secret_access_key="SECRET_ACCESS_KEY"

Environment Variable

In JSON/YAML format:

export AWS_S3='{type: s3, bucket: sling-bucket, access_key_id: ACCESS_KEY_ID, secret_access_key: "SECRET_ACCESS_KEY"}'

export AWS_S3='{type: s3, bucket: sling-bucket, profile: my-profile}'

# or via AWS classic environment variables
export AWS_ACCESS_KEY_ID='<aws_access_key_id>'
export AWS_SECRET_ACCESS_KEY='<aws_secret_access_key>'
export AWS_SESSION_TOKEN='<aws_session_token>'

Sling Env File YAML

connections:
  AWS_S3:
    type: s3
    bucket: <bucket>
    profile: <my-profile>

  AWS_S3_OTHER:
    type: s3
    bucket: <bucket>
    access_key_id: <access_key_id>
    secret_access_key: '<secret_access_key>'
PreviousStorage ConnectionsNextAzure Storage

Last updated 6 months ago

See to learn more about the sling env.yaml file.

If you are facing issues connecting, please reach out to us at , on or open a Github Issue .

here
support@slingdata.io
discord
here