AWS S3

Connect & Ingest data from / into a AWS S3 Bucket

Setup

The following credentials keys are accepted:

• bucket (required)

• use_environment (optional) -> whether to use the AWS environment variables to access. Accepts true or false.

• profile (optional) (or provide environment variable AWS_PROFILE)

• access_key_id (optional) (or provide environment variable AWS_ACCESS_KEY_ID)

• secret_access_key (optional) (or provide environment variable AWS_SECRET_ACCESS_KEY)

• session_token (optional) (or provide environment variable AWS_SESSION_TOKEN)

• region (optional) (or provide environment variable AWS_REGION)

• endpoint (optional) (or provide environment variable AWS_ENDPOINT)

• role_arn (optional) (or provide environment variable AWS_ROLE_ARN)

• encryption_algorithm (optional since v1.2.15) The server-side encryption algorithm to use. Accepts AES256, aws:kms or aws:kms:arn.

• encryption_kms_key (optional since v1.2.15) If encryption_algorithm is aws:kms or aws:kms:arn, this is the KMS key ID or ARN to use for encryption.

Using sling conns

Here are examples of setting a connection named AWS_S3. We must provide the type=s3 property:

$ sling conns set AWS_S3 type=s3 bucket=sling-bucket profile=my-profile

$ sling conns set AWS_S3 type=s3 bucket=sling-bucket access_key_id=ACCESS_KEY_ID secret_access_key="SECRET_ACCESS_KEY"

Environment Variable

In JSON/YAML format:

export AWS_S3='{type: s3, bucket: sling-bucket, access_key_id: ACCESS_KEY_ID, secret_access_key: "SECRET_ACCESS_KEY"}'

export AWS_S3='{type: s3, bucket: sling-bucket, profile: my-profile}'

# or via AWS classic environment variables
export AWS_ACCESS_KEY_ID='<aws_access_key_id>'
export AWS_SECRET_ACCESS_KEY='<aws_secret_access_key>'
export AWS_SESSION_TOKEN='<aws_session_token>'

Sling Env File YAML

See here to learn more about the sling env.yaml file.

connections:
  AWS_S3:
    type: s3
    bucket: <bucket>
    profile: <my-profile>

  AWS_S3_OTHER:
    type: s3
    bucket: <bucket>
    access_key_id: <access_key_id>
    secret_access_key: '<secret_access_key>'

If you are facing issues connecting, please reach out to us at [e-mail], on discord or open a Github Issue here.