Deploy With Github Actions

Use Github Actions to trigger your Sling Replications

Click here to see the Sling Run Action package to use in Github Actions.

Set Repository Secrets

We first need to set our connections credentials as repository secrets for Github Actions to access them securely.

For our example, we can create two envronment variable secrets: MY_POSTGRES and MY_SNOWFLAKE.

To see examples of setting environment variables for each type of connector, click below:

• File/Storage Connections (see here)

• Database Connections (see here)

Create Replication YAML

Let's create a file in folder replications, calling it postgres-snowflake.yaml. See here for details on configuring replications.

# We first need to make sure our connections are available in our environment
# See https://docs.slingdata.io/sling-cli/environment for help

source: MY_POSTGRES
target: MY_SNOWFLAKE

# default config options which apply to all streams
defaults:
  mode: full-refresh # valid choices: incremental, truncate, full-refresh, snapshot

  # specify pattern to use for object naming in target connection, see below for options
  object: '{target_schema}.{stream_schema}_{stream_table}'

  # source_options: # optional for more advanced options for source connection
  # target_options: # optional for more advanced options for target connection

streams:
  finance.accounts:
  finance.users:
    disabled: true
  finance.departments:
    object: '{target_schema}.finance_departments_old' # overwrite default object
    source_options:
      empty_as_null: false
  finance."Transactions":
    mode: incremental # overwrite default mode
    primary_key: id
    update_key: last_updated_at

Create Workflow

Now we can create our Github Actions workflow to run this on demand:

name: Sync Postgres to Snowflake

on:
  workflow_dispatch:

jobs:
  pg-snowflake:
    runs-on: ubuntu-20.04

    steps:
      - uses: actions/checkout@v2
        
      - name: Run Sling Replication
        uses: slingdata-io/sling-action@v2
        env:
          MY_POSTGRES: ${{ secrets.MY_POSTGRES }}
          MY_SNOWFLAKE: ${{ secrets.MY_SNOWFLAKE }}
        with:
          command: run -r replications/postgres-snowflake.yaml