Run With Github Actions
Use Github Actions to trigger your Sling Replications
We first need to set our connections credentials as repository secrets for Github Actions to access them securely.
For our example, we can create two envronment variable secrets:
MY_POSTGRES and MY_SNOWFLAKE.
To see examples of setting environment variables for each type of connector, click below:
Let's create a file in folder
replications, calling it postgres-snowflake.yaml. See here for details on configuring replications.# We first need to make sure our connections are available in our environment
# See https://docs.slingdata.io/sling-cli/environment for help
source: MY_POSTGRES
target: MY_SNOWFLAKE
# default config options which apply to all streams
defaults:
mode: full-refresh # valid choices: incremental, truncate, full-refresh, snapshot
# specify pattern to use for object naming in target connection, see below for options
object: '{target_schema}.{stream_schema}_{stream_table}'
# source_options: # optional for more advanced options for source connection
# target_options: # optional for more advanced options for target connection
streams:
finance.accounts:
finance.users:
disabled: true
finance.departments:
object: '{target_schema}.finance_departments_old' # overwrite default object
source_options:
empty_as_null: false
finance."Transactions":
mode: incremental # overwrite default mode
primary_key: id
update_key: last_updated_at
Now we can create our Github Actions workflow to run this on demand:
name: Sync Postgres to Snowflake
on:
workflow_dispatch:
jobs:
pg-snowflake:
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v2
- name: Run Sling Replication
uses: slingdata-io/sling-action@v1
env:
MY_POSTGRES: ${{ secrets.MY_POSTGRES }}
MY_SNOWFLAKE: ${{ secrets.MY_SNOWFLAKE }}
with:
command: run -r replications/postgres-snowflake.yaml