> For the complete documentation index, see [llms.txt](https://docs.slingdata.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.slingdata.io/sling-platform/platform/self-hosting/shared-mode.md). # High-Availability Agents Run many agent replicas on one key, with failover and platform-managed scaling By default, every agent process needs its own unique key (`single` mode). Agents can instead run in **shared mode**, where many replica processes connect with the **same key** and act as one logical agent: * The platform load-balances job executions across replicas (least-loaded first). * Heartbeats and metrics are aggregated across replicas in the UI. * If a replica dies mid-execution, the platform automatically **requeues the stalled execution** on a surviving replica (failover). There are two ways to run shared-mode agents: manually (you scale the replicas) or platform-managed (the platform provisions and scales them for you on Docker or Kubernetes). ## Enabling Shared Mode Shared mode is enabled on the **platform server**, not on the agent: ```yaml services: sling-platform: environment: SLING_RUNNER_MODE: shared ``` This is a server-level setting that applies to **every agent created afterwards** — existing agents keep their current mode. The platform is authoritative at connect time, so replicas need only the agent key; the agent's own `SLING_RUNNER_MODE` does not decide the mode. ## Manual Shared-Mode Replicas Run any number of agent processes with the same key. Setting `SLING_RUNNER_MODE=shared` on the agent is optional (the platform decides), but harmless:
agent-deployment.yaml (shared-mode replicas) ```yaml # kubernetes: a Deployment with replicas > 1 and one shared key env: - name: SLING_PLATFORM_HOST value: "http://sling-platform:7878" - name: SLING_AGENT_KEY valueFrom: secretKeyRef: name: sling-platform-secrets key: SLING_AGENT_KEY - name: SLING_RUNNER_MODE value: "shared" - name: POD_NAME # lets the platform display the pod name per replica valueFrom: fieldRef: fieldPath: metadata.name ```
The same works with `docker compose up --scale sling-agent=3` or several `docker run` invocations sharing one key. {% hint style="info" %} The agent must exist as **shared** on the platform side, which means it was created while `SLING_RUNNER_MODE=shared` was set on the platform server (agents created through the Docker (Local) or Kubernetes hosting options are always shared). Each connecting replica then registers its own session against the shared key. {% endhint %} {% hint style="warning" %} **All replicas of a shared key must run a recent agent binary.** Older binaries predate HA and subscribe to the broadcast subject without a queue group, so mixing them with new-style replicas on one key can deliver — and run — the same job twice. The platform rejects a session-less (old) binary that tries to join a key already held by a live session, and logs a warning when it sees one. Upgrade every replica before scaling out. {% endhint %} {% hint style="warning" %} **Each replica needs its own cache directory.** The agent keeps a local cache under its data dir; point it at per-pod ephemeral storage (an `emptyDir` volume in Kubernetes, the container filesystem in Docker) — **never** a shared `PersistentVolumeClaim` across replicas. This applies to replicas you run yourself; platform-managed agents (Docker (Local) and Kubernetes) already give each replica its own ephemeral cache. {% endhint %} ### Concurrency Limits in Shared Mode An agent's **max executions** is a **pool-wide** cap in shared mode: the platform compares it against the *aggregate* number of active executions across all replicas, not per replica. The default heuristic (`cpu_count × 2`) scales automatically, because the reported CPU count is summed across live replicas — adding a replica raises the pool's capacity without any configuration change. ## Docker-Local Managed Agents For single-host installs (the typical `docker compose up` deployment), the platform can provision agent replica containers directly on the local Docker daemon. Enable it on the **platform** container:
docker-compose.yaml (Docker-local runners) ```yaml services: sling-platform: image: slingdata/sling-platform environment: # ... required vars ... SLING_DOCKER_RUNNERS: '{"mode": "unix-socket", "network": "sling-net"}' volumes: - sling-platform-data:/home/sling - /var/run/docker.sock:/var/run/docker.sock # gives the platform access to the daemon networks: [sling-net] networks: sling-net: ```
Once enabled, a **Docker (Local)** hosting option appears in the agent creation dialog. The platform creates the containers, reconciles the replica count every 30 seconds, scales down oldest-first, and removes the containers when the agent is deleted. `SLING_DOCKER_RUNNERS` accepts a bare mode (`unix-socket` or `tcp`) or a JSON object. The `network` field is required when the master runs in a container, so replicas can reach it by container name. All fields except `mode`:
SLING_DOCKER_RUNNERS fields ```json { "mode": "unix-socket", "host": "unix:///var/run/docker.sock", "tls_verify": false, "cert_path": "", "image": "slingdata/sling:latest", "network": "sling-net", "labels": { "team": "data" }, "restart_policy": "unless-stopped", "cpu": 2, "memory_mb": 3072, "master_url": "", "nats_url": "" } ```
| Field | Default | Description | | -------------------------- | ------------------------------------ | --------------------------------------------------------------------------- | | `mode` | *(required)* | `unix-socket` or `tcp` | | `host` | `unix:///var/run/docker.sock` | Daemon address (`tcp://host:2376` for tcp mode) | | `tls_verify` / `cert_path` | *(off)* | TLS client certs for tcp mode | | `network` | *(empty)* | Docker network the replicas join; required when the master is containerized | | `image` | `slingdata/sling:` | Agent image override | | `cpu` | `2` | CPU limit per replica container | | `memory_mb` | `3072` | Memory limit (MB) per replica container | | `master_url` | *(derived)* | URL replicas use to reach the platform | | `nats_url` | *(derived)* | `ws://` NATS URL override | | `restart_policy` | `unless-stopped` | Restart policy for replica containers | | `labels` | *(empty)* | Extra container labels (JSON object) | {% hint style="warning" %} **Docker socket security**: mounting `/var/run/docker.sock` into the platform container gives it **root-equivalent access to the host**. For security-sensitive deployments, use [rootless Docker](https://docs.docker.com/engine/security/rootless/) or put a proxy like [tecnativa/docker-socket-proxy](https://github.com/Tecnativa/docker-socket-proxy) in front of the socket (allow only container/image endpoints) and point the platform at it with `SLING_DOCKER_RUNNERS='{"mode": "tcp", "host": "tcp://docker-socket-proxy:2375"}'`. {% endhint %} ## Kubernetes-Managed Agents (HA) When the platform runs in (or can reach) a Kubernetes cluster, it can manage agent replicas as a **Deployment** (plus a Secret for the key, and an optional HorizontalPodAutoscaler). Set up in this order. The platform tests the Kubernetes API at startup, so apply the RBAC **before** you restart it. ### 1. Apply the RBAC The platform needs permissions in the runner namespace (deployments, secrets, pods, horizontalpodautoscalers). Save the manifest below as `runner-rbac.yaml` and apply it: ```bash kubectl apply -f runner-rbac.yaml ``` It creates the runner namespace, the service account, a Role and a RoleBinding. {% hint style="warning" %} The manifest puts the service account in namespace `sling-platform`. If your platform runs in a different namespace, change it in **two** places: the `ServiceAccount` metadata and the `RoleBinding` subject. {% endhint %}
runner-rbac.yaml ```yaml apiVersion: v1 kind: Namespace metadata: name: sling-runners --- # Service account used by the sling-platform master pod apiVersion: v1 kind: ServiceAccount metadata: name: sling-platform namespace: sling-platform # the namespace of your platform deployment --- apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: sling-runner-manager namespace: sling-runners rules: - apiGroups: ["apps"] resources: ["deployments"] verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] - apiGroups: [""] resources: ["secrets"] verbs: ["get", "create", "update", "delete"] - apiGroups: [""] resources: ["pods"] verbs: ["get", "list", "watch"] - apiGroups: ["autoscaling"] resources: ["horizontalpodautoscalers"] verbs: ["get", "create", "update", "delete"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: sling-runner-manager namespace: sling-runners subjects: - kind: ServiceAccount name: sling-platform namespace: sling-platform # the namespace of your platform deployment roleRef: kind: Role name: sling-runner-manager apiGroup: rbac.authorization.k8s.io ```
To confirm the permissions are correct: ```bash kubectl auth can-i create deployments --namespace sling-runners \ --as system:serviceaccount:sling-platform:sling-platform ``` ### 2. Configure the platform Set the environment variable **and** the service account on the platform pod. Both are necessary — with `mode: incluster` the pod authenticates as its service account, and the default one has no permissions. ```yaml spec: serviceAccountName: sling-platform # must match the ServiceAccount above containers: - name: sling-platform env: - name: SLING_K8S_RUNNERS value: "incluster" # or "kubeconfig" when the platform runs outside the cluster ``` {% hint style="warning" %} **Replica pods must be able to reach the platform.** The platform gives them `SLING_PLATFORM_HOST` when it is not `localhost`. An external name such as `https://sling.example.com` often does not resolve inside the cluster. When that applies, set `master_url` and `nats_url` to the internal service DNS: ```json { "mode": "incluster", "master_url": "http://sling-platform..svc.cluster.local:7878", "nats_url": "ws://sling-platform..svc.cluster.local:4443" } ``` {% endhint %} ### 3. Restart the platform ```bash kubectl rollout restart deployment/sling-platform -n sling-platform kubectl rollout status deployment/sling-platform -n sling-platform ``` The platform tests the Kubernetes API at startup. If it starts and stays up, the connection is good. If the API is unreachable or the RBAC is wrong, the pod stops with an error in its logs. ### 4. Create the agent The configuration alone starts no pods. A **Kubernetes** hosting option now shows in the agent creation dialog, with: * **Desired replicas** — the platform reconciles the Deployment to this count every 30 seconds. * **Autoscale (HPA)** — optional; the platform creates an `autoscaling/v2` HorizontalPodAutoscaler with your min/max replicas and target CPU %, and stops managing the replica count itself. Requires [metrics-server](https://github.com/kubernetes-sigs/metrics-server) in the cluster. Install it before you use this option. Create an agent with Kubernetes hosting. The platform then creates the Deployment, the key Secret and the pods in the runner namespace. Each pod is named `sling-agent-`: ```bash kubectl get pods -n sling-runners ``` Agents created this way are always shared-mode, so `SLING_RUNNER_MODE` is not necessary for them. Each pod gets its own `emptyDir` cache at `/home/sling` — you do not configure storage. When you delete the agent, the platform removes the Deployment, the HorizontalPodAutoscaler and the Secret. `SLING_K8S_RUNNERS` accepts a bare mode (`incluster` or `kubeconfig`) or a JSON object. All fields except `mode`:
SLING_K8S_RUNNERS fields ```json { "mode": "incluster", "kubeconfig": "/etc/sling/kubeconfig", "namespace": "sling-runners", "image": "slingdata/sling:latest", "service_account": "", "node_selector": { "pool": "workers" }, "tolerations": [{ "key": "dedicated", "operator": "Equal", "value": "sling", "effect": "NoSchedule" }], "image_pull_secrets": ["regcred"], "resources": { "cpu_request": "500m", "cpu_limit": "2000m", "mem_request": "1Gi", "mem_limit": "3Gi" }, "master_url": "", "nats_url": "" } ```
| Field | Default | Description | | ----------------------------------------------- | ------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------- | | `mode` | *(required)* | `incluster` or `kubeconfig` | | `kubeconfig` | `$KUBECONFIG` or `~/.kube/config` | Kubeconfig path (kubeconfig mode) | | `namespace` | `sling-runners` | Namespace for runner Deployments | | `image` | `slingdata/sling:` | Agent image override | | `resources.cpu_request` / `resources.cpu_limit` | `500m` / `2000m` | CPU per replica pod | | `resources.mem_request` / `resources.mem_limit` | `1Gi` / `3Gi` | Memory per replica pod | | `master_url` | *(derived)* | URL replica pods use to reach the platform (defaults to the platform host or `http://sling-platform..svc.cluster.local:`) | | `nats_url` | *(derived)* | `ws://` NATS URL override | | `image_pull_secrets` | *(empty)* | Image pull secret names (JSON array) | | `node_selector` | *(empty)* | Pod nodeSelector (JSON object) | | `tolerations` | *(empty)* | Pod tolerations (JSON array) | | `service_account` | *(empty)* | Service account for the runner pods | {% hint style="info" %} With `SLING_DOCKER_RUNNERS` or `SLING_K8S_RUNNERS` set, the platform verifies connectivity to the Docker daemon / Kubernetes API **at startup** and fails fast with a clear error if unreachable. {% endhint %} --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://docs.slingdata.io/sling-platform/platform/self-hosting/shared-mode.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.