Modes

Here are the various loading modes available. All modes load into a new temporary table prior to final load.

Mode	Description
full-refresh	This is the default mode. The target table will be dropped and recreated with the source data.
incremental	The source data will be merged or appended into the target table. If the table does not exist, it will be created. See below for more details.
truncate	Similar to full-refresh, except that the target table is truncated instead of dropped. This keeps any special DDL / GRANT applied.
snapshot	Appends the full dataset with an added timestamp column. If the target table exists, Sling will insert into / append data with a _sling_loaded_at column. If it does not, the table will be created.
backfill	Similar to incremental, but takes a range input to backfill a specific update_key range, such as dates or numbers.

Incremental Mode Strategies

Load Strategy	Primary Key	Update Key	Stream Strategy
New Data Upsert (update/insert)	yes	yes	Only new records after max(update_key)
Full Data Upsert (update/insert)	yes	no	Full data
Append Only (insert, no update)	no	yes	Only new records after max(update_key)

Incremental or Backfill Mode With Custom SQL

When using incremental or backfill mode with a custom SQL stream, Sling provides two placeholder options to handle incremental loading:

• {incremental_where_cond}: Injects a complete WHERE condition

• {incremental_value}: Injects only the value, allowing for custom condition logic

Using {incremental_where_cond}

This placeholder injects a complete WHERE condition based on the update_key:

SELECT * FROM my_schema.my_table 
WHERE {incremental_where_cond}

Sling will replace the placeholder as follows:

• First run (table doesn't exist): WHERE 1=1

• Subsequent runs: WHERE my_update_key > '[incremental_value]'

For example, if the last maximum value was '2001-01-01 01:01:01', the query becomes:

SELECT * FROM my_schema.my_table 
WHERE my_update_key > '2001-01-01 01:01:01'

Using {incremental_value}

This placeholder gives you more control over the WHERE condition by injecting only the value:

SELECT * FROM my_schema.my_table 
WHERE my_int_key > coalesce({incremental_value}, 0)  -- For numeric columns
-- or
WHERE my_timestamp > coalesce({incremental_value}, '2001-01-01')  -- For timestamp columns

Sling will replace the placeholder as follows:

• First run (table doesn't exist): {incremental_value} becomes null

• Subsequent runs: {incremental_value} becomes the last maximum value

For example, if the last maximum value was 99, the query becomes:

SELECT * FROM my_schema.my_table 
WHERE my_int_key > coalesce(99, 0)