AWS S3

Connect & Ingest data from / into a AWS S3 Bucket

Setup

The following credentials keys are accepted:

  • bucket (required)

  • profile (optional) (or provide environment variable AWS_PROFILE)

  • access_key_id (optional) (or provide environment variable AWS_ACCESS_KEY_ID)

  • secret_access_key (optional) (or provide environment variable AWS_SECRET_ACCESS_KEY)

  • session_token (optional) (or provide environment variable AWS_SESSION_TOKEN)

  • region (optional) (or provide environment variable AWS_REGION)

  • endpoint (optional) (or provide environment variable AWS_ENDPOINT)

  • role_arn (optional) (or provide environment variable AWS_ROLE_ARN)

  • encryption_algorithm (optional since v1.2.15) The server-side encryption algorithm to use. Accepts AES256, aws:kms or aws:kms:arn.

  • encryption_kms_key (optional since v1.2.15) If encryption_algorithm is aws:kms or aws:kms:arn, this is the KMS key ID or ARN to use for encryption.

Using sling conns

Here are examples of setting a connection named AWS_S3. We must provide the type=s3 property:

$ sling conns set AWS_S3 type=s3 bucket=sling-bucket profile=my-profile

$ sling conns set AWS_S3 type=s3 bucket=sling-bucket access_key_id=ACCESS_KEY_ID secret_access_key="SECRET_ACCESS_KEY"

Environment Variable

In JSON/YAML format:

export AWS_S3='{type: s3, bucket: sling-bucket, access_key_id: ACCESS_KEY_ID, secret_access_key: "SECRET_ACCESS_KEY"}'

export AWS_S3='{type: s3, bucket: sling-bucket, profile: my-profile}'

# or via AWS classic environment variables
export AWS_ACCESS_KEY_ID='<aws_access_key_id>'
export AWS_SECRET_ACCESS_KEY='<aws_secret_access_key>'
export AWS_SESSION_TOKEN='<aws_session_token>'

Sling Env File YAML

See here to learn more about the sling env.yaml file.

connections:
  AWS_S3:
    type: s3
    bucket: <bucket>
    profile: <my-profile>

  AWS_S3_OTHER:
    type: s3
    bucket: <bucket>
    access_key_id: <access_key_id>
    secret_access_key: '<secret_access_key>'

If you are facing issues connecting, please reach out to us at support@slingdata.io, on discord or open a Github Issue here.

Last updated