ADBC (Arrow)

Connect & Ingest data from / to multiple databases via Arrow Database Connectivity (ADBC)

Arrow Database Connectivity (ADBC) provides a standardized interface for accessing various databases using the Apache Arrow columnar format. ADBC enables efficient, high-performance data movement with zero-copy semantics and native Arrow support (v1.5.2+)

Supported Database Types

Sling supports the following databases via ADBC drivers:

• PostgreSQL - Full support via ADBC PostgreSQL driver

• MySQL - Support via ADBC MySQL driver (v9.4+)

• SQL Server - Full support via ADBC SQL Server driver

• Snowflake - Full support via ADBC Snowflake driver

• SQLite - Support via ADBC SQLite driver

• DuckDB - Full support via ADBC DuckDB driver

• BigQuery - Full support via ADBC BigQuery driver

• Trino - Support via ADBC Trino driver (v4.0+)

Setup

Before using ADBC connections, you need to install the appropriate ADBC driver using the dbc CLI tool (see details here):

# Install drivers for your target databases
dbc install postgresql
dbc install mysql
dbc install trino
dbc install snowflake
# ... etc

Drivers are automatically installed to:

• macOS: ~/Library/Application Support/ADBC/Drivers

• Linux: ~/.local/share/ADBC/Drivers

• Windows: %LOCALAPPDATA%\ADBC\Drivers

Sling will automatically discover installed drivers from these locations.

Enabling ADBC on Connections

To use ADBC with your existing database connections, simply add use_adbc: true to your connection configuration. This allows you to keep your original connection properties and format while enabling ADBC's high-performance data transfer.

Connection Properties

The following ADBC-specific properties can be added to any supported database connection:

• use_adbc (optional) -> Enable ADBC driver for this connection (true or false). Default is false.

• adbc_uri (optional) -> Override the automatically constructed ADBC URI. Sling automatically builds the ADBC URI from your connection properties, but you can specify a custom URI if needed.

Database-Specific Examples

PostgreSQL with ADBC

connections:
  POSTGRES:
    type: postgres
    host: localhost
    user: myuser
    password: mypass
    database: mydatabase
    port: 5432
    use_adbc: true
    # adbc_uri: "postgresql://myuser:mypass@localhost:5432/mydatabase"  # optional override

ADBC URI format: postgresql://user:password@host:port/database

Official Documentation: Apache ADBC PostgreSQL Driver

MySQL with ADBC

connections:
  MYSQL:
    type: mysql
    host: localhost
    user: root
    password: mypass
    database: mydb
    port: 3306
    use_adbc: true
    # adbc_uri: "root@tcp(localhost:3306)/mydb"  # optional override

ADBC URI format: user@tcp(host:port)/database

Official Documentation: Apache ADBC MySQL Driver

SQL Server with ADBC

connections:
  MSSQL:
    type: sqlserver
    host: localhost
    user: sa
    password: mypass
    database: master
    port: 1433
    use_adbc: true
    # adbc_uri: "mssql://sa:mypass@localhost:1433/master"  # optional override

ADBC URI format: mssql://user:password@host:port/database

Official Documentation: Apache ADBC SQL Server Driver

Snowflake with ADBC

connections:
  SNOWFLAKE:
    type: snowflake
    account: myaccount.us-east-1
    user: myuser
    password: mypass
    database: mydb
    schema: myschema
    use_adbc: true
    # adbc_uri: "snowflake://myuser:mypass@myaccount/mydb/myschema"  # optional override

ADBC URI format: snowflake://user:password@account/database/schema

Official Documentation: Apache ADBC Snowflake Driver

DuckDB with ADBC

connections:
  DUCKDB:
    type: duckdb
    instance: /path/to/file.db
    use_adbc: true
    # adbc_uri: "duckdb:///path/to/file.db"  # optional override

ADBC URI format: duckdb:///path/to/file.db or duckdb://:memory:

Official Documentation: Apache ADBC DuckDB Driver

SQLite with ADBC

connections:
  SQLITE:
    type: sqlite
    instance: /path/to/file.db
    use_adbc: true
    # adbc_uri: "sqlite:///path/to/file.db"  # optional override

ADBC URI format: sqlite:///path/to/file.db

Official Documentation: Apache ADBC SQLite Driver

BigQuery with ADBC

connections:
  BIGQUERY:
    type: bigquery
    project: myproject
    dataset: mydataset
    key_file: /path/to/service.account.json
    use_adbc: true
    # adbc_uri: "bigquery://myproject"  # optional override

ADBC URI format: bigquery://project

Official Documentation: Apache ADBC BigQuery Driver

Trino with ADBC

connections:
  TRINO:
    type: trino
    http_url: "http://myuser@localhost:8080?catalog=hive&schema=default"
    use_adbc: true
    # adbc_uri: "http://myuser@localhost:8080?catalog=hive&schema=default"  # optional override

ADBC URI format: http://user@host:port?catalog=catalog&schema=schema

Official Documentation: Apache ADBC Trino Driver

Using sling conns

Here are examples of enabling ADBC on existing connections:

# PostgreSQL with ADBC
$ sling conns set POSTGRES type=postgres host=localhost user=myuser password=mypass database=mydb use_adbc=true

# MySQL with ADBC
$ sling conns set MYSQL type=mysql host=localhost user=root password=mypass database=mydb use_adbc=true

# SQL Server with ADBC
$ sling conns set MSSQL type=sqlserver host=localhost user=sa password=mypass database=master use_adbc=true

# Snowflake with ADBC
$ sling conns set SNOWFLAKE type=snowflake account=myaccount user=myuser password=mypass database=mydb use_adbc=true

# With custom ADBC URI override
$ sling conns set POSTGRES type=postgres host=localhost user=myuser password=mypass database=mydb use_adbc=true adbc_uri="postgresql://myuser:mypass@localhost:5432/mydb"

Environment Variable

export POSTGRES='{ type: postgres, host: localhost, user: myuser, password: mypass, database: mydb, use_adbc: true }'

export MYSQL='{ type: mysql, host: localhost, user: root, password: mypass, database: mydb, use_adbc: true }'

export SNOWFLAKE='{ type: snowflake, account: myaccount, user: myuser, password: mypass, database: mydb, use_adbc: true }'

Sling Env File YAML

See here to learn more about the sling env.yaml file.

connections:
  # PostgreSQL with ADBC
  POSTGRES:
    type: postgres
    host: localhost
    user: myuser
    password: mypass
    database: mydatabase
    use_adbc: true

  # MySQL with ADBC
  MYSQL:
    type: mysql
    host: localhost
    user: root
    password: mypass
    database: mydb
    use_adbc: true

  # SQL Server with ADBC
  MSSQL:
    type: sqlserver
    host: localhost
    user: sa
    password: mypass
    database: master
    use_adbc: true

  # Snowflake with ADBC
  SNOWFLAKE:
    type: snowflake
    account: myaccount
    user: myuser
    password: mypass
    database: mydb
    schema: myschema
    use_adbc: true

  # DuckDB with ADBC
  DUCKDB:
    type: duckdb
    instance: /path/to/file.db
    use_adbc: true

  # With custom ADBC URI override
  POSTGRES_CUSTOM:
    type: postgres
    host: localhost
    user: myuser
    password: mypass
    database: mydatabase
    use_adbc: true
    adbc_uri: "postgresql://myuser:mypass@localhost:5432/mydatabase?sslmode=require"

Additional Resources

• Apache ADBC Project

• ADBC Driver Documentation

• Apache Arrow Documentation

If you are facing issues connecting, please reach out to us at [email protected], on discord or open a Github Issue here.