CLI Pro

Use CLI Pro to enable advanced features

CLI Pro Features

Sling CLI Pro extends the core functionality with advanced features designed for production environments and complex data operations.

• ✅ Parallel streams & Retries

• ✅ Capture Deletes (similar to CDC)

• ✅ File Target Incremental Mode

• ✅ State Based Incremental

• ✅ Chunked Backfill

• ✅ Constraints & Checks

• ✅ Hooks (such as http, query, check and more)

• ✅ Pipelines to run multiple steps in sequence

• ✅ Support Sling and its continuous development

You can obtain a token for free at https://dash.slingdata.io. There is 7-day trial (no credit card needed).

Once you have a token, just put the value into the SLING_CLI_TOKEN environment variable before running sling (make sure the version is 1.4+).

For Pricing details see here.

Parallel Processing & Retries

Optimize performance with parallel stream processing:

• Run multiple streams concurrently

• Automatic retry mechanisms for failed operations

• Configurable concurrency and retry settings

All you have to do is set the environment variables:

• SLING_THREADS sets the maximum number of concurrent stream runs. Accepts an integer value, default is 1.

• SLING_RETRIES sets the maximum number of retries for a failed stream run. Accepts an integer value, default is 0.

source: ...
target: ...

streams:
  ...

env:
  SLING_THREADS: 3   # maximum of 3 streams concurrently
  SLING_RETRIES: 1   # maximum of 1 retry per failed stream

Chunked Backfill

Process large datasets efficiently with automatic chunking:

• Break down backfill ranges into manageable chunks

• Support for time-based (hours, days, months) and numeric chunks

• Parallel processing of chunks for improved performance

streams:
  my_schema.orders:
    update_key: order_date
    source_options:
      range: '2023-01-01,2023-12-31'
      chunk_size: 7d  # Process in 7-day chunks

See Database to Database Backfill for detailed examples.

File & State Based Incremental Loading

Maintain state across file loads with intelligent incremental processing:

• Track and resume file processing from last successful position

• Support for incremental writes to databases and files

• Automatic file partitioning and truncation management

See Database to File Incremental Loading and File to Database Incremental Loading for detailed examples.

Capture Deletes (CDC)

Track deleted records using a deleted_at column:

• Automatically detect and mark deleted records

• Maintain historical record states

• Support for soft deletes in target systems

See Delete Missing Records for implementation details.

Constraints

Enforce data quality with constraint checking:

• Column-level validation rules

• Custom SQL-like constraints

• Automatic constraint verification before loading

See Constraints for implementation examples.

Hooks

Extend functionality with pre and post-processing hooks:

• Execute custom logic before/after operations

• Support for shell commands and SQL scripts

• Integration with external tools and workflows

• Types of hooks supported:

  • Query Hook
  • HTTP Hook
  • Check Hook
  • Inspect Hook
  • Upload Hook
  • Download Hook
  • Delete Hook
  • Log Hook

See Hooks for usage examples and patterns.

Pipelines

A Pipeline in Sling allows you to execute multiple steps in sequence. Each step can be a different type of operation (same as hooks), enabling you to create complex workflows by chaining together various actions like running replications, executing queries, making HTTP requests, and more.

See Pipelines for usage examples and patterns.

Frequently Asked Questions

How many subscriptions do I need?

Each CLI Pro subscription includes 2 tokens:

• 1 Production token: For use in production environments

• 1 Development token: For development and testing

We recommend purchasing one subscription per team or project, regardless of your deployment method. This allows you to:

• Use the production token across your production environments (whether permanent servers or ephemeral containers)

• Share the development token among team members for testing and development

For example:

• A data engineering team handling customer data → 1 subscription

• A separate analytics team handling reporting → 1 subscription

• Multiple teams in an organization → 1 subscription per team

• Consultancy/Freelancer with multiple customers → 1 subscription per customer

Please use tokens responsibly and in accordance with our Terms of Use.