ScyllaDB

Connect & Ingest data from / to a ScyllaDB database

Setup

The following credentials keys are accepted:

• host (required) -> The hostname / ip of the instance (contact point)

• user (optional) -> The username to access the instance (alias: username)

• password (optional) -> The password to access the instance

• port (optional) -> The port of the instance. Default is 9042.

• keyspace (optional) -> The default keyspace to use (aliases: database, schema)

• hosts (optional) -> Additional contact points (comma-separated hostnames or host:port) for multi-node clusters

• local_dc (optional) -> Local datacenter name for DC-aware + token-aware routing (alias: datacenter). Recommended for multi-node / multi-DC clusters.

• consistency (optional) -> Default write/read consistency. Examples: ONE, LOCAL_ONE, QUORUM, LOCAL_QUORUM, ALL. Default is driver default (QUORUM).

• timeout (optional) -> Query timeout in seconds. Default is 15.

• connect_timeout (optional) -> Connection setup timeout in seconds. Defaults to timeout.

• disable_initial_host_lookup (optional) -> If true (default), do not look up peer hosts from the control connection. Useful for Docker/Kubernetes where broadcast addresses are not reachable. Set false on multi-node bare-metal clusters for peer discovery / failover.

• num_conns (optional) -> Number of connections per host. Default is 2.

• page_size (optional) -> CQL page size for reads. Default is 5000.

• insert_concurrency (optional) -> Concurrent insert workers for bulk writes. Default is 16.

• tls (optional) -> whether to use TLS for connecting (true / false / skip-verify)

• cert_file (optional) -> the client certificate to use to access the instance via TLS (file path or raw)

• cert_key_file (optional) -> the client key to use to access the instance via TLS (file path or raw)

• cert_ca_file (optional) -> the client CA certificate to use to access the instance via TLS (file path or raw)

ScyllaDB is Cassandra-compatible and uses the native CQL protocol (via gocql / Scylla fork). Tables require a primary key; if none is provided on write, Sling will pick a suitable column (preferring columns named id). Prefer setting primary_key explicitly in replications.

When Sling auto-creates a keyspace, it uses NetworkTopologyStrategy with replication_factor: 1 (fine for single-node dev). For production, create keyspaces yourself with the correct per-DC replication factors.

Using sling conns

Here are examples of setting a connection named SCYLLADB. We must provide the type=scylladb property:

$ sling conns set SCYLLADB type=scylladb host=<host> user=<user> password=<password> port=9042 keyspace=<keyspace>

# Multi-node cluster with local DC
$ sling conns set SCYLLADB type=scylladb host=10.0.0.1 hosts=10.0.0.2,10.0.0.3 \
    user=<user> password=<password> keyspace=<keyspace> local_dc=us-east-1 \
    consistency=LOCAL_QUORUM disable_initial_host_lookup=false

# OR use url
$ sling conns set SCYLLADB url="scylladb://myuser:[e-mail]:9042/mykeyspace"

Environment Variable

See here to learn more about the .env.sling file.

export SCYLLADB='scylladb://myuser:[e-mail]:9042/mykeyspace'
export SCYLLADB='{ type: scylladb, user: "myuser", password: "mypass", host: "host.ip", port: 9042, keyspace: "mykeyspace" }'
export SCYLLADB_CLUSTER='{ type: scylladb, host: "10.0.0.1", hosts: "10.0.0.2,10.0.0.3", user: "myuser", password: "mypass", keyspace: "analytics", local_dc: "us-east-1", consistency: "LOCAL_QUORUM", disable_initial_host_lookup: false }'

Sling Env File YAML

See here to learn more about the sling env.yaml file.

connections:
  SCYLLADB:
    type: scylladb
    host: host.ip
    port: 9042
    user: <user>
    password: <password>
    keyspace: <keyspace>

  SCYLLADB_CLUSTER:
    type: scylladb
    host: 10.0.0.1
    hosts: 10.0.0.2,10.0.0.3
    port: 9042
    user: <user>
    password: <password>
    keyspace: analytics
    local_dc: us-east-1
    consistency: LOCAL_QUORUM
    disable_initial_host_lookup: false
    timeout: 30
    insert_concurrency: 32

  SCYLLADB_URL:
    type: scylladb
    url: "scylladb://myuser:[e-mail]:9042/mykeyspace"

  SCYLLADB_TLS:
    type: scylladb
    host: host.ip
    port: 9042
    user: <user>
    password: <password>
    keyspace: <keyspace>
    tls: true

Examples

source: scylladb
target: snowflake

defaults:
  mode: full-refresh
  object: public.{stream_schema}_{stream_table}

streams:
  mykeyspace.events:
  mykeyspace.users:
    select: [id, email, created_at]

source: postgres
target: scylladb

defaults:
  mode: full-refresh
  object: analytics.{stream_table}
  # ScyllaDB requires a primary key on every table
  primary_key: [id]

streams:
  public.orders:
  public.order_items:
    primary_key: [order_id, item_id]

# Incremental load into Scylla (INSERT is upsert by primary key)
source: s3
target: scylladb

defaults:
  mode: incremental
  object: analytics.{stream_table}
  primary_key: [id]
  update_key: updated_at

streams:
  s3://bucket/events/*.parquet:

Notes

• No SQL views — Scylla supports materialized views with strict rules, not regular CREATE VIEW. Prefer tables as sources.

• CQL is not SQL — arbitrary SQL (joins, aliases, subqueries) is not supported as a stream. Use table names or simple CQL.

• Merge strategies — the default merge strategy is insert (CQL INSERT upserts by primary key). SQL-style update/delete merge strategies are not supported.

• Keyspaces — create production keyspaces yourself with the correct NetworkTopologyStrategy RF per DC.

If you are facing issues connecting, please reach out to us at [e-mail], on discord or open a Github Issue here.