# Dynamic Endpoints

Dynamic endpoints allow you to programmatically generate multiple endpoint configurations based on runtime data. This is powerful for APIs where the list of available endpoints or resources isn't known until you query the API itself.

## When to Use Dynamic Endpoints

Use dynamic endpoints when:

* The list of available resources/endpoints is determined at runtime
* You need to generate similar endpoints for multiple entities (e.g., one endpoint per table, per user, per organization)
* Endpoint configuration depends on data fetched from the API
* You want to avoid manually listing hundreds of similar endpoints

## Static vs. Dynamic Endpoints

### Static Endpoints (Standard)

```yaml
endpoints:
  users:
    request:
      url: "{state.base_url}/users"
    response:
      records:
        jmespath: "data[]"

  orders:
    request:
      url: "{state.base_url}/orders"
    response:
      records:
        jmespath: "data[]"

  products:
    request:
      url: "{state.base_url}/products"
    response:
      records:
        jmespath: "data[]"
```

**Limitations:**

* Must know all endpoints in advance
* Repetitive configuration for similar endpoints
* Manual updates needed when new resources are added

### Dynamic Endpoints (Advanced)

```yaml
dynamic_endpoints:
  - setup:
      # Fetch list of available tables
      - request:
          url: "{state.base_url}/tables"
        response:
          processors:
            - expression: 'jmespath(response.json, "tables")'
              output: "state.table_list"
              aggregation: last

    iterate: "state.table_list"
    into: "state.table_name"

    endpoint:
      name: "{state.table_name}"
      request:
        url: "{state.base_url}/tables/{state.table_name}/data"
      response:
        records:
          jmespath: "data[]"
```

**Benefits:**

* Automatically discovers available endpoints
* Single configuration for many similar endpoints
* Adapts automatically to API changes

## Dynamic Endpoint Structure

A dynamic endpoint definition has these parts:

```yaml
dynamic_endpoints:
  - setup: [<array of sequence calls>]    # Optional: Get data for iteration
    iterate: <expression or state variable> # What to iterate over
    into: <variable name>                  # Variable to store current item
    endpoint: <endpoint configuration>     # Template for generated endpoints
```

### Properties

| Property   | Required | Description                                                    | Example                                                            |
| ---------- | -------- | -------------------------------------------------------------- | ------------------------------------------------------------------ |
| `setup`    | No       | Sequence of calls to prepare data                              | Fetch list of resources                                            |
| `iterate`  | Yes      | Expression, JSON literal, or inline list / object to loop over | `state.table_list`, `'["a","b"]'`, or a raw YAML array (see below) |
| `into`     | Yes      | Variable name for current item                                 | `state.table_name`                                                 |
| `endpoint` | Yes      | Endpoint configuration template                                | Standard endpoint config                                           |

### Forms of `iterate`

`iterate` accepts four shapes; pick whichever is clearest for your use case:

1. **JMESPath expression string** — evaluated against state. Use when the list is built dynamically during `setup` (or comes from `defaults.state`).

   ```yaml
   iterate: 'state.config.resources[].name'
   ```
2. **JSON literal string** — a one-line inline array. Useful for small static lists.

   ```yaml
   iterate: '["users", "orders", "products"]'
   ```
3. **Inline YAML array (recommended for nested objects)** — write the list directly as YAML, with full support for nested fields. Each element is bound to the `into` variable as-is, so the template can reference fields with dot notation (e.g. `{state.report.name}`).

   ```yaml
   dynamic_endpoints:
     - iterate:
         - name: 'daily_signups'
           dimensions: [date, country, plan]
         - name: 'weekly_revenue'
           dimensions: [week, product_id, currency]
       into: 'state.report'
       endpoint:
         name: '{state.report.name}'
         request:
           url: '{state.base_url}/reports/{state.report.name}'
           parameters:
             dimensions: '{join(state.report.dimensions, ",")}'
         response:
           records:
             jmespath: 'rows[]'
   ```
4. **Inline YAML single object** — treated as a one-element list. Useful when you only have one item but want the same templating contract.

   ```yaml
   iterate:
     name: 'daily_signups'
     dimensions: [date, country, plan]
   into: 'state.report'
   ```

## Examples

### Example 1: Database Tables

Dynamically create endpoints for each table in a database API:

```yaml
name: "Database API"
description: "API for accessing database tables"

defaults:
  state:
    base_url: "https://api.example.com/v1"

dynamic_endpoints:
  - setup:
      # Get list of all tables
      - request:
          url: "{state.base_url}/metadata/tables"
        response:
          processors:
            - expression: 'jmespath(response.json, "tables[].name")'
              output: "state.available_tables"
              aggregation: collect

    # Create one endpoint per table
    iterate: "state.available_tables"
    into: "state.table_name"

    endpoint:
      name: "table_{state.table_name}"
      description: "Data from {state.table_name} table"

      request:
        url: "{state.base_url}/tables/{state.table_name}"
        parameters:
          limit: 1000

      pagination:
        next_state:
          offset: "{state.offset + 1000}"
        stop_condition: "length(response.records) < 1000"

      response:
        records:
          jmespath: "rows[]"
          primary_key: ["id"]
```

**Result:** If the API returns tables `["users", "orders", "products"]`, Sling automatically creates three endpoints:

* `table_users`
* `table_orders`
* `table_products`

### Example 2: Multi-Organization Data

Create endpoints for each organization the user has access to:

```yaml
name: "Multi-Org API"
description: "API with per-organization endpoints"

defaults:
  state:
    base_url: "https://api.example.com/v2"
    # Default date range for event queries
    start_date: '{date_format(date_add(now(), -30, "day"), "%Y-%m-%d")}'
    end_date: '{date_format(now(), "%Y-%m-%d")}'

authentication:
  type: oauth2
  flow: client_credentials
  client_id: "{secrets.client_id}"
  client_secret: "{secrets.client_secret}"
  authentication_url: "{state.base_url}/oauth/token"

dynamic_endpoints:
  - setup:
      # Get list of organizations user can access
      - request:
          url: "{state.base_url}/user/organizations"
        response:
          processors:
            - expression: 'jmespath(response.json, "organizations")'
              output: "state.org_list"
              aggregation: last

    # Create one endpoint per organization
    iterate: "state.org_list"
    into: "state.current_org"

    endpoint:
      name: "org_{state.current_org.id}_events"
      description: "Events for organization: {state.current_org.name}"

      request:
        url: "{state.base_url}/organizations/{state.current_org.id}/events"
        parameters:
          from_date: "{state.start_date}"
          to_date: "{state.end_date}"

      response:
        records:
          jmespath: "events[]"
          primary_key: ["event_id"]

        processors:
          # Add organization context to each record
          - expression: "state.current_org.id"
            output: "record.organization_id"
          - expression: "state.current_org.name"
            output: "record.organization_name"
```

### Example 3: Geographic Regions

Create endpoints for each geographic region:

```yaml
name: "Regional Sales API"
description: "Sales data by region"

defaults:
  state:
    base_url: "https://api.example.com"
    regions:
      - code: "us-east"
        name: "US East Coast"
      - code: "us-west"
        name: "US West Coast"
      - code: "eu-west"
        name: "Europe West"
      - code: "apac"
        name: "Asia Pacific"

dynamic_endpoints:
  # No setup needed - iterate over predefined regions
  - iterate: "state.regions"
    into: "state.region"

    endpoint:
      name: "sales_{state.region.code}"
      description: "Sales data for {state.region.name}"

      request:
        url: "{state.base_url}/regions/{state.region.code}/sales"
        parameters:
          date_from: "{state.start_date}"
          date_to: "{state.end_date}"

      pagination:
        next_state:
          page: "{state.page + 1}"
        stop_condition: 'jmespath(response.json, "has_more") == false'

      response:
        records:
          jmespath: "sales[]"
          primary_key: ["sale_id"]

        processors:
          # Tag each record with region info
          - expression: "state.region.code"
            output: "record.region_code"
          - expression: "state.region.name"
            output: "record.region_name"
```

### Example 4: Inline YAML Array with Nested Objects

When the list of resources is known up-front and each entry needs more than just a name (e.g. a report name plus the dimensions to pull), use a raw YAML array directly in `iterate`. No `setup` call, no JSON-string escaping — the YAML is the data.

```yaml
name: "Analytics Reports API"

defaults:
  state:
    base_url: "https://api.example.com/v2"

dynamic_endpoints:
  - iterate:
      - name: 'daily_signups'
        dimensions: [date, country, plan]
        granularity: day
      - name: 'weekly_revenue'
        dimensions: [week, product_id, currency]
        granularity: week
      - name: 'monthly_churn'
        dimensions: [month, segment, reason]
        granularity: month
    into: 'state.report'

    endpoint:
      name: '{state.report.name}'
      description: 'Analytics report: {state.report.name}'

      request:
        url: '{state.base_url}/reports/{state.report.name}'
        parameters:
          dimensions: '{join(state.report.dimensions, ",")}'
          granularity: '{state.report.granularity}'

      response:
        records:
          jmespath: 'rows[]'
          primary_key: ['id']
```

**Result:** three endpoints — `daily_signups`, `weekly_revenue`, `monthly_churn` — each fetching only the dimensions declared inline. To add another report, append one more YAML entry; no code or setup step changes.

## How Dynamic Endpoints Work

```mermaid
graph TD
    A[Start API Spec Loading] --> B{Has dynamic_endpoints?}
    B -->|No| C[Load Static Endpoints]
    B -->|Yes| D[Execute Setup Sequence]

    D --> E[Evaluate iterate Expression]
    E --> F[Get List of Items]

    F --> G{For Each Item}
    G --> H[Set into Variable]
    H --> I[Render Endpoint Template]
    I --> J[Add to Endpoint List]

    J --> G
    G -->|Done| K[Combine with Static Endpoints]
    C --> K

    K --> L[Final Endpoint List]

    style A fill:#4a9eff,stroke:#ffffff,stroke-width:2px,color:#ffffff
    style D fill:#ff8c42,stroke:#ffffff,stroke-width:2px,color:#ffffff
    style E fill:#ffd54f,stroke:#ffffff,stroke-width:2px,color:#000000
    style I fill:#7cb342,stroke:#ffffff,stroke-width:2px,color:#ffffff
    style L fill:#ab47bc,stroke:#ffffff,stroke-width:2px,color:#ffffff
```

## Execution Flow

1. **Authentication**: API authenticates once
2. **Setup Phase** (if defined):
   * Executes setup sequence calls
   * Fetches data needed for iteration
   * Stores results in state
3. **Iteration Phase**:
   * Evaluates the `iterate` expression
   * For each item, sets the `into` variable
   * Renders the endpoint template with current values
   * Adds the generated endpoint to the list
4. **Endpoint Registration**:
   * Combines dynamic and static endpoints
   * Applies defaults
   * Validates all endpoints
5. **Execution**: Endpoints run normally (can be selected via patterns)

## Variable Scoping

When rendering dynamic endpoints, variables are available in this order of precedence:

1. **Current iteration value** (`into` variable) - Highest priority
2. **State variables** from setup
3. **Environment variables**
4. **Secrets**
5. **Defaults**

```yaml
dynamic_endpoints:
  - setup:
      - request:
          url: "{state.base_url}/config"
        response:
          processors:
            - expression: 'jmespath(response.json, "api_version")'
              output: "state.api_version"  # Available to endpoint template
              aggregation: last

    iterate: '["users", "orders", "products"]'
    into: "state.resource_type"  # Available as {state.resource_type} in template

    endpoint:
      name: "{state.resource_type}_v{state.api_version}"  # Uses both
      request:
        url: "{env.BASE_URL}/{state.resource_type}"  # Environment variable
        headers:
          Authorization: "Bearer {secrets.api_token}"  # Secret
```

## Combining Static and Dynamic Endpoints

You can mix static and dynamic endpoints in the same spec:

```yaml
name: "Hybrid API"

# Static endpoints
endpoints:
  health_check:
    request:
      url: "{state.base_url}/health"

  metadata:
    request:
      url: "{state.base_url}/metadata"

# Dynamic endpoints
dynamic_endpoints:
  - iterate: '["users", "orders", "products"]'
    into: "state.entity"
    endpoint:
      name: "{state.entity}"
      request:
        url: "{state.base_url}/{state.entity}"
```

**Result:** 5 total endpoints:

* `health_check` (static)
* `metadata` (static)
* `users` (dynamic)
* `posts` (dynamic)
* `comments` (dynamic)

## Selecting Dynamic Endpoints

When running, you can select dynamic endpoints just like static ones:

```bash
# Run all endpoints (static + dynamic)
sling conns test MY_API

# Run specific dynamic endpoint(s)
sling conns test MY_API --endpoints table_users

# Run pattern matching
sling conns test MY_API --endpoints "table_*"

# Run multiple
sling conns test MY_API --endpoints "org_123_*,org_456_*"
```

## Limitations and Considerations

### Performance Considerations

1. **Setup Overhead**: Setup sequence runs once before endpoint generation
2. **Large Lists**: Many dynamic endpoints increase memory usage
3. **Discovery Time**: Fetching endpoint list adds latency

### Best Practices

#### 1. Filter in Setup

Don't generate endpoints you won't use:

```yaml
setup:
  - request:
      url: "{state.base_url}/tables"
    response:
      processors:
        # Only include tables matching pattern
        - expression: >
            filter(jmespath(response.json, "tables"), "starts_with(name, 'prod_')")
          output: "state.filtered_tables"
          aggregation: last
```

#### 2. Use Meaningful Names

Make generated endpoint names descriptive:

```yaml
# Good: Clear what this endpoint does
name: "region_{region.code}_sales_daily"

# Avoid: Unclear names
name: "ep_{index}"
```

#### 3. Add Metadata

Include helpful information in generated endpoints:

```yaml
endpoint:
  name: "{state.table_name}"
  description: "Data from {state.table_name} table (schema: {table_schema})"
  docs: "https://docs.example.com/tables/{state.table_name}"
```

#### 4. Validate Iteration Data

Ensure the iteration data is valid:

```yaml
setup:
  - request:
      url: "{state.base_url}/tables"
    response:
      processors:
        - expression: 'jmespath(response.json, "tables")'
          output: "state.tables"
          aggregation: last
```

## Troubleshooting

### No Endpoints Generated

Check that:

1. Setup sequence succeeds
2. Iterate expression returns non-empty array
3. Into variable is properly referenced in endpoint template

Use debug logging:

```yaml
setup:
  - request:
      url: "{state.base_url}/resources"
    response:
      processors:
        - expression: 'jmespath(response.json, "resources")'
          output: "state.resource_list"
          aggregation: last
        # Debug: Log what we got
        - expression: log("Found " + string(length(state.resource_list)) + " resources")
          output: ""
```

### Templates Not Rendering

Verify variable names match:

```yaml
# ❌ Wrong: Mismatched variable names
iterate: "state.items"
into: "state.item"
endpoint:
  name: "{state.resource}"  # Should be {item}

# ✓ Correct: Matching variable names
iterate: "state.items"
into: "state.item"
endpoint:
  name: "{state.item}"
```

### Duplicate Endpoint Names

Ensure each generated endpoint has a unique name:

```yaml
# ❌ Wrong: Same name for all
endpoint:
  name: "data_endpoint"  # All will have same name!

# ✓ Correct: Unique names
endpoint:
  name: "data_{state.resource_id}"  # Each gets unique name
```

> 💡 **Tip:** While dynamic endpoints is powerful, start with static endpoints for simpler APIs. Use dynamic endpoints when you have many similar endpoints or when endpoint discovery is necessary.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.slingdata.io/concepts/api-specs/dynamic-endpoints.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.