Functions
Unlock the power of your data workflows with these versatile built-in functions. They enable sophisticated data transformations, validations, and manipulations, and can be seamlessly integrated into your pipelines, hooks, and transforms.
CLI Pro Required: Functions require a CLI Pro token or Platform Plan.
String Functions
contains(string, substring)
Checks if string contains substring
string, substring
Boolean
contains("hello world", "world") → true
join(array, separator)
Joins array elements into string
array, separator
String
join(["a", "b", "c"], ", ") → "a, b, c"
length(string|array|map)
Gets length of string/array/map
value: String, array or map
Number
length("hello") → 5, length([1,2]) → 2
lower(string)
Converts string to lowercase
string: Input string
Lowercase string
lower("HELLO") → "hello"
snake(string)
Converts string to snake_case (spaces become underscores)
string: Input string
snake_case string
snake("Hello World") → "hello_world"
replace(string, pattern, replace)
Replaces occurrences of pattern
string, pattern, replacement
Modified string
replace("hello", "l", "x") → "hexxo"
split_part(string, sep, index)
Gets part of split string by index
string, separator, index (0-based)
String part
split_part("a,b,c", ",", 1) → "b"
split(string, separator)
Splits string into array
string, separator
Array of strings
split("a,b,c", ",") → ["a", "b", "c"]
substring(string, start[, end])
Extracts part of a string (0-indexed)
string: Input string
start: Start index
end: Optional end index
Substring
substring("hello world", 0, 5) → "hello"
trim(string)
Removes whitespace from start/end
string: Input string
Trimmed string
trim(" hello ") → "hello"
upper(string)
Converts string to uppercase
string: Input string
Uppercase string
upper("hello") → "HELLO"
parse_ms_uuid(string)
Parses Microsoft UUID format
string: 16-byte UUID string
Parsed UUID
parse_ms_uuid(binary_uuid) → "12345678-..."
replace_0x00(string)
Removes null bytes from string
string: Input string
Cleaned string
replace_0x00("hello\x00world") → "helloworld"
remove_diacritics(string)
Removes accents/diacritics
string: Input string
ASCII string
remove_diacritics("café") → "cafe"
replace_non_printable(string)
Removes non-printable characters
string: Input string
Cleaned string
replace_non_printable("hello\x01world") → "helloworld"
Numeric Functions
bool_parse(value)
Converts value to boolean
value: Value to convert
Boolean or error
bool_parse("true") → true
float_format(value, format)
Formats float (Go format)
value: Number
format: Format string
Formatted string
float_format(42.5678, "%.2f") → "42.57"
float_parse(value)
Converts value to float
value: Value to convert
Float or error
float_parse("42.5") → 42.5
greatest(array|val1, val2...)
Finds maximum value
array or multiple values
Maximum value
greatest(1, 5, 3) → 5
int_format(value, format)
Formats integer (Go format)
value: Number
format: Format string
Formatted string
int_format(42, "%05d") → "00042"
int_parse(value)
Converts value to integer
value: Value to convert
Integer or error
int_parse("42") → 42
int_range(start, end[, step])
Generates integer range
start, end: Integers, step: Optional integer (default 1)
Array of integers
int_range(1, 5) → [1,2,3,4,5]
is_greater(val1, val2)
Checks if val1 > val2
val1, val2: Values to compare
Boolean
is_greater(5, 3) → true
is_less(val1, val2)
Checks if val1 < val2
val1, val2: Values to compare
Boolean
is_less(3, 5) → true
least(array|val1, val2...)
Finds minimum value
array or multiple values
Minimum value
least(1, 5, 3) → 1
Date Functions
Uses Go's time package and strftime conventions via timefmt-go. Please refer to man 3 strftime and man 3 strptime for formatter syntax.
now()
Gets current date and time
None
Time object
now()
date_parse(string[, format])
Parses string to time object
string: Date string
format: "auto" or strftime
Time object or error
date_parse("2022-01-01T10:00:00Z", "auto")
date_format(date, format)
Formats time object to string
date: Time object
format: strftime format
Formatted string
date_format(now(), "%Y-%m-%d") → "2023-10-27" (example date)
date_add(date, duration[, unit])
Adds duration to time object
date, duration (int), unit (string, default "s")
Time object
date_add(now(), -7, "day")
date_diff(date1, date2[, unit])
Time between dates
date1, date2, unit (string, default "s")
Number (float)
date_diff(date_add(now(), 1, "day"), now(), "hour") → 24.0
date_trunc(date, unit)
Truncates date to unit start
date, unit ("year", "month", "day", "hour", etc.)
Time object
date_trunc(now(), "month") → First day of current month at 00:00:00
date_extract(date, part)
Extracts part from date
date, part ("year", "month", "day", "hour", etc.)
Number
date_extract(now(), "year") → 2023 (example year)
date_last(date[, period])
Gets last day of period
date, period ("month", "year", default "month")
Time object
date_last(now()) → Last day of current month
date_first(date[, period])
Gets first day of period
date, period ("month", "year", default "month")
Time object
date_first(now()) → First day of current month
date_range(start, end[, step])
Creates array of dates
start, end: Dates, step: Duration string or int+unit
Array of dates
date_range("2023-01-01", "2023-01-03", "1d") → ["2023-01-01", "2023-01-02", "2023-01-03"]
date_timezone(date, timezone)
Converts time to specified IANA Timezone
date: Time object
timezone: Timezone string (e.g., "UTC", "America/New_York")
Time object
date_timezone(now(), "America/New_York") → Time in NY timezone
Date function unit/part/period parameters often accept: "year", "month", "week", "day", "hour", "minute", "second". range function with dates requires time objects as start/end.
Value Handling Functions
cast(value, type)
Converts value to type
value, type ("int", "float", "string", "bool")
Converted value or error
cast("42", "int") → 42
coalesce(val1, val2, ...)
Returns first non-null value
Multiple values
First non-null value
coalesce(null, sync.val, state.val, "default")
element(array, index)
Gets element at 0-based index
array, index (integer)
Element or error
element(["a", "b"], 1) → "b"
equals(val1, val2)
Checks deep equality
val1, val2
Boolean
equals(response.status, 200) → true
first_valid(val1, val2, ...)
Returns first non-null and non-empty value
Multiple values
First valid value
first_valid("", state.val, "default") → state.val (if not empty)
if(condition, then, else)
Conditional expression
condition (bool), then_val, else_val
Selected value
if(state.count > 0, "has items", "empty")
is_empty(value)
Checks if value is empty
value (string, array, map)
Boolean
is_empty("") → true, is_empty([]) → true
is_null(value)
Checks if value is null
value
Boolean
is_null(state.optional_param) → true or false
require(val[, error_msg])
Ensures value is not null or error
val, error_msg (optional)
Value or error
require(secrets.api_key, "API Key is required")
try_cast(value, type)
Tries conversion, returns null
value, type
Converted value or null
try_cast("abc", "int") → null
Collection Functions
array(val1, val2, ...)
Creates array from values
Multiple values
Array
array(1, 2, 3) → [1, 2, 3]
chunk(array|queue, size)
Splits array/queue into chunks
array or queue.name, size (int)
Channel of arrays
Used in iterate: over: chunk(queue.ids, 50)
filter(array, expression)
Filters array using expression
array, expression (string condition)
Filtered array
filter([1, 2, 3], "element > 1") → [2, 3]
get_path(object, path)
Gets value using dot notation
object, path (string, e.g., "a.b[0]")
Value at path
get_path(response.json, "user.profile.email")
jmespath(object, expression)
Evaluates JMESPath expression
object, expression (string)
Query result
jmespath(response.json, "data.items[?age > 30]")
keys(map)
Gets keys from map
map object
Array of keys
keys({"a": 1, "b": 2}) → ["a", "b"]
exists(collection, item)
Checks if key exists in map or value exists in array
collection: Map or array, item: Key or value to find
Boolean
exists({"a": 1}, "a") → true, exists([1, 2], 2) → true
object(k1, v1, k2, v2, ...)
Creates object from key/value pairs
Even number of arguments (key, value pairs)
Map/object
object("name", "John", "age", 30) → {"name": "John", "age": 30}
pluck(data, column_name)
Extracts values from a column
data: Array of maps, column_name: String
Array of values
pluck([{"name": "John", "age": 30}, {"name": "Jane", "age": 25}], "name") → ["John", "Jane"]
range(start, end[, step])
Generates range (auto-detects type)
start, end: Numbers or dates, step: Optional
Array or channel
range(1, 5) → [1,2,3,4,5], delegates to int_range or date_range
sort(array[, descending])
Sorts array elements
array, descending (optional bool)
Sorted array
sort([3, 1, 2]) → [1, 2, 3]
values(map)
Gets values from map
map object
Array of values
values({"a": 1, "b": 2}) → [1, 2]
object_rename(map, old, new, ...)
Renames keys in map
map: Object, followed by pairs of old_key, new_key
Modified map
object_rename({"a": 1, "b": 2}, "a", "x") → {"x": 1, "b": 2}
object_delete(map, key1, ...)
Deletes keys from map
map: Object, followed by keys to delete
Modified map
object_delete({"a": 1, "b": 2}, "a") → {"b": 2}
object_casing(map, casing)
Transforms keys in map to specified casing
map: Object, casing: "snake", "camel", "upper", "lower"
Modified map
object_casing({"firstName": "John"}, "snake") → {"first_name": "John"}
object_merge(map1, map2, ...)
Merges multiple maps together
Two or more maps to merge (later maps override earlier)
Merged map
object_merge({"a": 1}, {"b": 2}, {"c": 3}) → {"a": 1, "b": 2, "c": 3}
Encoding/Decoding Functions
encode_url(string)
URL encodes a string
string
Encoded string
encode_url("a b") → "a%20b"
decode_url(string)
URL decodes a string
string
Decoded string
decode_url("a%20b") → "a b"
encode_base64(string)
Base64 encodes string
string
Encoded string
encode_base64("user:pass") → "dXNlcjpwYXNz"
decode_base64(string)
Base64 decodes string
string
Decoded string
decode_base64("dXNlcjpwYXNz") → "user:pass"
hash(string[, algorithm])
Creates hash of string
string, algorithm ("md5", "sha1", "sha256", default "md5")
Hash string (hex)
hash("hello", "md5") → "5d4..."
json_parse(string)
Parses JSON string
string: JSON string
Object/Array/Value
json_parse('{"name": "John"}') → {"name": "John"}
Utility Functions
uuid()
Generates random UUID v4
None
UUID string
uuid() → "..."
log(message)
Logs a message during evaluation
message
The message (passthru)
log("Processing record: " + record.id)
regex_match(string, pattern)
Checks if string matches pattern
string, pattern (Go regex)
Boolean
regex_match("img_123.jpg", "^img_.*\\.jpg$") → true
regex_extract(string, pattern[, idx])
Extracts matches using regex
string, pattern, idx (optional)
Matches/group or null
regex_extract("id=123", "id=(\\d+)", 1) → "123"
regex_replace(string, pattern, replacement)
Replaces pattern matches
string, pattern (Go regex), replacement
Modified string
regex_replace("hello123world", "\\d+", "-") → "hello-world"
pretty_table(data)
Renders array of maps as table
data: Array of maps
Formatted table string
pretty_table([{"name": "John", "age": 30}]) → formatted table
type_of(value)
Gets type of value
value
Type string
type_of(42) → "integer"
conn_property(connection, key)
Gets property value from connection
connection: Connection name (string)
key: Property name (string)
Property value
conn_property("my_db", "host") → "localhost"
machine_stats()
Gets machine resource statistics
None
Object with stats
machine_stats() → {"memory_percent": 45.2, "cpu_percent": 23.5}
Last updated
Was this helpful?