# Dynamic Endpoints

Dynamic endpoints allow you to programmatically generate multiple endpoint configurations based on runtime data. This is powerful for APIs where the list of available endpoints or resources isn't known until you query the API itself.

## When to Use Dynamic Endpoints

Use dynamic endpoints when:

* The list of available resources/endpoints is determined at runtime
* You need to generate similar endpoints for multiple entities (e.g., one endpoint per table, per user, per organization)
* Endpoint configuration depends on data fetched from the API
* You want to avoid manually listing hundreds of similar endpoints

## Static vs. Dynamic Endpoints

### Static Endpoints (Standard)

```yaml
endpoints:
  users:
    request:
      url: "{state.base_url}/users"
    response:
      records:
        jmespath: "data[]"

  orders:
    request:
      url: "{state.base_url}/orders"
    response:
      records:
        jmespath: "data[]"

  products:
    request:
      url: "{state.base_url}/products"
    response:
      records:
        jmespath: "data[]"
```

**Limitations:**

* Must know all endpoints in advance
* Repetitive configuration for similar endpoints
* Manual updates needed when new resources are added

### Dynamic Endpoints (Advanced)

```yaml
dynamic_endpoints:
  - setup:
      # Fetch list of available tables
      - request:
          url: "{state.base_url}/tables"
        response:
          processors:
            - expression: 'jmespath(response.json, "tables")'
              output: "state.table_list"
              aggregation: last

    iterate: "state.table_list"
    into: "state.table_name"

    endpoint:
      name: "{state.table_name}"
      request:
        url: "{state.base_url}/tables/{state.table_name}/data"
      response:
        records:
          jmespath: "data[]"
```

**Benefits:**

* Automatically discovers available endpoints
* Single configuration for many similar endpoints
* Adapts automatically to API changes

## Dynamic Endpoint Structure

A dynamic endpoint definition has these parts:

```yaml
dynamic_endpoints:
  - setup: [<array of sequence calls>]    # Optional: Get data for iteration
    iterate: <expression or state variable> # What to iterate over
    into: <variable name>                  # Variable to store current item
    endpoint: <endpoint configuration>     # Template for generated endpoints
```

### Properties

| Property   | Required | Description                       | Example                                     |
| ---------- | -------- | --------------------------------- | ------------------------------------------- |
| `setup`    | No       | Sequence of calls to prepare data | Fetch list of resources                     |
| `iterate`  | Yes      | Expression or data to loop over   | `"state.table_list"` or `"['a', 'b', 'c']"` |
| `into`     | Yes      | Variable name for current item    | `"state.table_name"`                        |
| `endpoint` | Yes      | Endpoint configuration template   | Standard endpoint config                    |

## Examples

### Example 1: Database Tables

Dynamically create endpoints for each table in a database API:

```yaml
name: "Database API"
description: "API for accessing database tables"

defaults:
  state:
    base_url: "https://api.example.com/v1"

dynamic_endpoints:
  - setup:
      # Get list of all tables
      - request:
          url: "{state.base_url}/metadata/tables"
        response:
          processors:
            - expression: 'jmespath(response.json, "tables[].name")'
              output: "state.available_tables"
              aggregation: collect

    # Create one endpoint per table
    iterate: "state.available_tables"
    into: "state.table_name"

    endpoint:
      name: "table_{state.table_name}"
      description: "Data from {state.table_name} table"

      request:
        url: "{state.base_url}/tables/{state.table_name}"
        parameters:
          limit: 1000

      pagination:
        next_state:
          offset: "{state.offset + 1000}"
        stop_condition: "length(response.records) < 1000"

      response:
        records:
          jmespath: "rows[]"
          primary_key: ["id"]
```

**Result:** If the API returns tables `["users", "orders", "products"]`, Sling automatically creates three endpoints:

* `table_users`
* `table_orders`
* `table_products`

### Example 2: Multi-Organization Data

Create endpoints for each organization the user has access to:

```yaml
name: "Multi-Org API"
description: "API with per-organization endpoints"

defaults:
  state:
    base_url: "https://api.example.com/v2"
    # Default date range for event queries
    start_date: '{date_format(date_add(now(), -30, "day"), "%Y-%m-%d")}'
    end_date: '{date_format(now(), "%Y-%m-%d")}'

authentication:
  type: oauth2
  flow: client_credentials
  client_id: "{secrets.client_id}"
  client_secret: "{secrets.client_secret}"
  authentication_url: "{state.base_url}/oauth/token"

dynamic_endpoints:
  - setup:
      # Get list of organizations user can access
      - request:
          url: "{state.base_url}/user/organizations"
        response:
          processors:
            - expression: 'jmespath(response.json, "organizations")'
              output: "state.org_list"
              aggregation: last

    # Create one endpoint per organization
    iterate: "state.org_list"
    into: "state.current_org"

    endpoint:
      name: "org_{state.current_org.id}_events"
      description: "Events for organization: {state.current_org.name}"

      request:
        url: "{state.base_url}/organizations/{state.current_org.id}/events"
        parameters:
          from_date: "{state.start_date}"
          to_date: "{state.end_date}"

      response:
        records:
          jmespath: "events[]"
          primary_key: ["event_id"]

        processors:
          # Add organization context to each record
          - expression: "state.current_org.id"
            output: "record.organization_id"
          - expression: "state.current_org.name"
            output: "record.organization_name"
```

### Example 3: Geographic Regions

Create endpoints for each geographic region:

```yaml
name: "Regional Sales API"
description: "Sales data by region"

defaults:
  state:
    base_url: "https://api.example.com"
    regions:
      - code: "us-east"
        name: "US East Coast"
      - code: "us-west"
        name: "US West Coast"
      - code: "eu-west"
        name: "Europe West"
      - code: "apac"
        name: "Asia Pacific"

dynamic_endpoints:
  # No setup needed - iterate over predefined regions
  - iterate: "state.regions"
    into: "state.region"

    endpoint:
      name: "sales_{state.region.code}"
      description: "Sales data for {state.region.name}"

      request:
        url: "{state.base_url}/regions/{state.region.code}/sales"
        parameters:
          date_from: "{state.start_date}"
          date_to: "{state.end_date}"

      pagination:
        next_state:
          page: "{state.page + 1}"
        stop_condition: 'jmespath(response.json, "has_more") == false'

      response:
        records:
          jmespath: "sales[]"
          primary_key: ["sale_id"]

        processors:
          # Tag each record with region info
          - expression: "state.region.code"
            output: "record.region_code"
          - expression: "state.region.name"
            output: "record.region_name"
```

## How Dynamic Endpoints Work

{% @mermaid/diagram content="graph TD
A\[Start API Spec Loading] --> B{Has dynamic\_endpoints?}
B -->|No| C\[Load Static Endpoints]
B -->|Yes| D\[Execute Setup Sequence]

```
D --> E[Evaluate iterate Expression]
E --> F[Get List of Items]

F --> G{For Each Item}
G --> H[Set into Variable]
H --> I[Render Endpoint Template]
I --> J[Add to Endpoint List]

J --> G
G -->|Done| K[Combine with Static Endpoints]
C --> K

K --> L[Final Endpoint List]

style A fill:#4a9eff,stroke:#ffffff,stroke-width:2px,color:#ffffff
style D fill:#ff8c42,stroke:#ffffff,stroke-width:2px,color:#ffffff
style E fill:#ffd54f,stroke:#ffffff,stroke-width:2px,color:#000000
style I fill:#7cb342,stroke:#ffffff,stroke-width:2px,color:#ffffff
style L fill:#ab47bc,stroke:#ffffff,stroke-width:2px,color:#ffffff" %}
```

## Execution Flow

1. **Authentication**: API authenticates once
2. **Setup Phase** (if defined):
   * Executes setup sequence calls
   * Fetches data needed for iteration
   * Stores results in state
3. **Iteration Phase**:
   * Evaluates the `iterate` expression
   * For each item, sets the `into` variable
   * Renders the endpoint template with current values
   * Adds the generated endpoint to the list
4. **Endpoint Registration**:
   * Combines dynamic and static endpoints
   * Applies defaults
   * Validates all endpoints
5. **Execution**: Endpoints run normally (can be selected via patterns)

## Variable Scoping

When rendering dynamic endpoints, variables are available in this order of precedence:

1. **Current iteration value** (`into` variable) - Highest priority
2. **State variables** from setup
3. **Environment variables**
4. **Secrets**
5. **Defaults**

```yaml
dynamic_endpoints:
  - setup:
      - request:
          url: "{state.base_url}/config"
        response:
          processors:
            - expression: 'jmespath(response.json, "api_version")'
              output: "state.api_version"  # Available to endpoint template
              aggregation: last

    iterate: '["users", "orders", "products"]'
    into: "state.resource_type"  # Available as {state.resource_type} in template

    endpoint:
      name: "{state.resource_type}_v{state.api_version}"  # Uses both
      request:
        url: "{env.BASE_URL}/{state.resource_type}"  # Environment variable
        headers:
          Authorization: "Bearer {secrets.api_token}"  # Secret
```

## Combining Static and Dynamic Endpoints

You can mix static and dynamic endpoints in the same spec:

```yaml
name: "Hybrid API"

# Static endpoints
endpoints:
  health_check:
    request:
      url: "{state.base_url}/health"

  metadata:
    request:
      url: "{state.base_url}/metadata"

# Dynamic endpoints
dynamic_endpoints:
  - iterate: '["users", "orders", "products"]'
    into: "state.entity"
    endpoint:
      name: "{state.entity}"
      request:
        url: "{state.base_url}/{state.entity}"
```

**Result:** 5 total endpoints:

* `health_check` (static)
* `metadata` (static)
* `users` (dynamic)
* `posts` (dynamic)
* `comments` (dynamic)

## Selecting Dynamic Endpoints

When running, you can select dynamic endpoints just like static ones:

```bash
# Run all endpoints (static + dynamic)
sling conns test MY_API

# Run specific dynamic endpoint(s)
sling conns test MY_API --endpoints table_users

# Run pattern matching
sling conns test MY_API --endpoints "table_*"

# Run multiple
sling conns test MY_API --endpoints "org_123_*,org_456_*"
```

## Limitations and Considerations

### Performance Considerations

1. **Setup Overhead**: Setup sequence runs once before endpoint generation
2. **Large Lists**: Many dynamic endpoints increase memory usage
3. **Discovery Time**: Fetching endpoint list adds latency

### Best Practices

#### 1. Filter in Setup

Don't generate endpoints you won't use:

```yaml
setup:
  - request:
      url: "{state.base_url}/tables"
    response:
      processors:
        # Only include tables matching pattern
        - expression: >
            filter(jmespath(response.json, "tables"), "starts_with(name, 'prod_')")
          output: "state.filtered_tables"
          aggregation: last
```

#### 2. Use Meaningful Names

Make generated endpoint names descriptive:

```yaml
# Good: Clear what this endpoint does
name: "region_{region.code}_sales_daily"

# Avoid: Unclear names
name: "ep_{index}"
```

#### 3. Add Metadata

Include helpful information in generated endpoints:

```yaml
endpoint:
  name: "{state.table_name}"
  description: "Data from {state.table_name} table (schema: {table_schema})"
  docs: "https://docs.example.com/tables/{state.table_name}"
```

#### 4. Validate Iteration Data

Ensure the iteration data is valid:

```yaml
setup:
  - request:
      url: "{state.base_url}/tables"
    response:
      processors:
        - expression: 'jmespath(response.json, "tables")'
          output: "state.tables"
          aggregation: last
```

## Troubleshooting

### No Endpoints Generated

Check that:

1. Setup sequence succeeds
2. Iterate expression returns non-empty array
3. Into variable is properly referenced in endpoint template

Use debug logging:

```yaml
setup:
  - request:
      url: "{state.base_url}/resources"
    response:
      processors:
        - expression: 'jmespath(response.json, "resources")'
          output: "state.resource_list"
          aggregation: last
        # Debug: Log what we got
        - expression: log("Found " + string(length(state.resource_list)) + " resources")
          output: ""
```

### Templates Not Rendering

Verify variable names match:

```yaml
# ❌ Wrong: Mismatched variable names
iterate: "state.items"
into: "state.item"
endpoint:
  name: "{state.resource}"  # Should be {item}

# ✓ Correct: Matching variable names
iterate: "state.items"
into: "state.item"
endpoint:
  name: "{state.item}"
```

### Duplicate Endpoint Names

Ensure each generated endpoint has a unique name:

```yaml
# ❌ Wrong: Same name for all
endpoint:
  name: "data_endpoint"  # All will have same name!

# ✓ Correct: Unique names
endpoint:
  name: "data_{state.resource_id}"  # Each gets unique name
```

> 💡 **Tip:** While dynamic endpoints is powerful, start with static endpoints for simpler APIs. Use dynamic endpoints when you have many similar endpoints or when endpoint discovery is necessary.