> For the complete documentation index, see [llms.txt](https://docs.slingdata.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.slingdata.io/concepts/replication/modes.md). # Modes Here are the various loading modes available. All modes load into a new temporary table prior to final load.

Mode	Description
`full-refresh`	This is the default mode. The target table will be dropped and recreated with the source data.
`incremental`	The source data will be merged or appended into the target table. If the table does not exist, it will be created. See below for more details.
`truncate`	Similar to `full-refresh`, except that the target table is truncated instead of dropped. This keeps any special DDL / GRANT applied.
`snapshot`	Appends the full dataset with an added timestamp column. If the target table exists, Sling will insert into / append data with a `_sling_loaded_at` column. If it does not, the table will be created.
`backfill`	Similar to `incremental`, but takes a `range` input to backfill a specific `update_key` range, such as dates or numbers.
`definition-only`	Creates the target table or file structure without transferring any data. For database targets, the table is created with the inferred schema but 0 rows. For file targets, only Parquet and Arrow formats are supported, creating a file with proper column types but no data rows.
`change-capture`	Captures row-level changes (inserts, updates, deletes) from the source database's transaction log. Performs an automatic initial snapshot on the first run, then reads incremental changes on subsequent runs. Requires a `primary_key`. See Change Capture (CDC) for details.

### Incremental Mode Strategies

Load Strategy	Primary Key	Update Key	Stream Strategy
New Data Upsert (update/insert)	`yes`	`yes`	Only new records after `max(update_key)`
Full Data Upsert (update/insert)	`yes`	no	Full data
Append Only (insert, no update)	no	`yes`	Only new records after `max(update_key)`

### First Run & Bootstrap What happens on the very first `incremental` run, when the target table is empty or does not exist: 1. **The target table is auto-created.** If the target does not exist, Sling creates it with column types inferred from the source data. You do not pre-create the table. 2. **The first run reads the full source table.** With no target rows and no stored watermark, there is no `max(update_key)` to filter on, so Sling reads every row (the incremental `WHERE` is effectively `1=1`). It then records `max(update_key)` as the watermark. Every later run pulls only rows where `update_key > watermark`. 3. **`primary_key` and `table_keys` are applied on that create.** The keys you set (`primary_key`, plus any `target_options.table_keys` such as an index or partition) are applied when Sling creates the table, not on a later run. An empty-but-existing target behaves the same as a missing one: with no rows, `max(update_key)` is null, so the first run reads the full source and seeds the watermark from it. {% hint style="info" %} This is why the first incremental run is as heavy as a full-refresh — it reads the whole table once to establish the watermark. Later runs are incremental. To skip the initial full load (for example when the target is already populated by another process), see [seeding the state](https://github.com/slingdata-io/sling-docs/tree/master/concepts/troubleshooting.md#skip-the-initial-full-load). {% endhint %} ### Watermark Safety (late-arriving data) When a strategy pulls "only new records after `max(update_key)`", Sling computes the watermark as `max(update_key)` in the **target** table and pulls source rows where `update_key > watermark`. This is a boundary condition to understand: * A row that lands in the source **with an `update_key` value at or below the current watermark** (for example a late-arriving event, or a backdated correction) is **not** picked up by the next incremental run — its key is not greater than the last maximum. * With a **primary key** set, rows whose key *is* greater than the watermark are upserted (updated in place). Rows below the watermark are simply not selected, so no update happens for them. To catch late data, add a **lookback / buffer** by writing a custom-SQL stream with the `{incremental_value}` placeholder and subtracting an interval from the boundary: ```sql -- re-scan the last 3 days on every run to absorb late arrivals SELECT * FROM my_schema.my_table WHERE updated_at > coalesce({incremental_value}, '2001-01-01')::timestamp - interval '3 days' ``` With a primary key, the re-scanned rows are upserted, so re-reading them is idempotent (no duplicates). Choose the interval to cover your worst-case lateness; a wider buffer re-reads more rows each run. See [Incremental or Backfill Mode With Custom SQL](#incremental-or-backfill-mode-with-custom-sql) below for the placeholder details. ### Incremental or Backfill Mode With Custom SQL When using `incremental` or `backfill` mode with a custom SQL stream, Sling provides two placeholder options to handle incremental loading: * `{incremental_where_cond}`: Injects a complete WHERE condition * `{incremental_value}`: Injects only the value, allowing for custom condition logic #### Using {incremental\_where\_cond} This placeholder injects a complete WHERE condition based on the `update_key`: ```sql SELECT * FROM my_schema.my_table WHERE {incremental_where_cond} ``` Sling will replace the placeholder as follows: * First run (table doesn't exist): `WHERE 1=1` * Subsequent runs: `WHERE my_update_key > '[incremental_value]'` For example, if the last maximum value was '2001-01-01 01:01:01', the query becomes: ```sql SELECT * FROM my_schema.my_table WHERE my_update_key > '2001-01-01 01:01:01' ``` #### Using {incremental\_value} This placeholder gives you more control over the WHERE condition by injecting only the value: ```sql SELECT * FROM my_schema.my_table WHERE my_int_key > coalesce({incremental_value}, 0) -- For numeric columns -- or WHERE my_timestamp > coalesce({incremental_value}, '2001-01-01') -- For timestamp columns ``` Sling will replace the placeholder as follows: * First run (table doesn't exist): `{incremental_value}` becomes `null` * Subsequent runs: `{incremental_value}` becomes the last maximum value For example, if the last maximum value was 99, the query becomes: ```sql SELECT * FROM my_schema.my_table WHERE my_int_key > coalesce(99, 0) ``` --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.slingdata.io/concepts/replication/modes.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.