Global Variables
Learn how to use Global Environment Variables with Sling
Sling utilizes the following global environment variables to further configure the load behavior. You can simply define them in your environment, the env.yaml file or the env section in a task or replication.
SLING_HOME_DIR
The sling home directory, which contains env.yaml. Will use default if not provided.
SLING_LOADED_AT_COLUMN
Whether to add an audit timestamp column named _sling_loaded_at in target object. Accepts values true, false, unix (for epoch integer values) or timestamp. true defaults to unix.
SLING_STREAM_URL_COLUMN
If source is file, whether to add a column _sling_stream_url with the source file path / url in target object. To enable, set to true.
SLING_TIMEOUT
The maximum number of minutes the sling replication should run. Once reached, it will kill the process. To enable, set an integer (SLING_TIMEOUT=10)
SLING_RECURSIVE_LIMIT
The number limit of file names to pull, when listing from cloud file systems such as S3, GCP and Azure Storage.
SLING_ROW_ID_COLUMN
Whether to add a column named _sling_row_id in the target object, which will have a random UUIDv7 value. This will be unique. To enable, set to true.
SLING_ROW_NUM_COLUMN
If source is file, whether to add a column named _sling_row_num in the target object, which will be the row number of the stream (incremented by record processed). To enable, set to true.
SLING_EXEC_ID_COLUMN
Whether to add a column named _sling_exec_id in the target object, which will have the run / execution string (a random UUIDv7 value). This will be unique per run. To enable, set to true.
SLING_STATE
The location to read/write information such as incremental values. Proper input format is CONN_NAME/key. For example: AWS_S3/my/folder or MY_SFTP/my/folder. Only storage connections are currently supported.
SLING_ALLOW_EMPTY
This is useful to create tables / files using the stream columns structure, even if there is no data. To enable, set to true.
SLING_DIRECT_INSERT
Tells sling to insert directly into the final table (not create a temp table before). To enable, set to true.
SLING_THREADS
sets the maximum number of concurrent stream runs. Accepts an integer value, default is 1.
SLING_RETRIES
sets the maximum number of retries for a failed stream run. Accepts an integer value, default is 0.
SLING_KEEP_TEMP
Tells sling to keep any temporary files or tables created in the load process. To enable, set to true
SLING_ENV_YAML
Provide the body of the env.yaml file as an environment variable.
SLING_DISABLE_TELEMETRY
this disables any anonymous usage reporting. These are used to improve sling. To disable, set this to true.
SLING_SHOW_PROGRESS
Whether the progress of the stream should be displayed (true or false).
SLING_LOGGING
How sling formats the log lines. Accepts values JSON, NO_COLOR or CONSOLE (default).
SLING_SAMPLE_SIZE
The number of records to process in order to infer column types (especially for file sources). Default is 900.
Last updated