# API Specs

Sling API specs are YAML files that define how to interact with REST APIs. They provide a structured way to specify authentication methods, define endpoints, configure request parameters, handle pagination, process responses, manage state for incremental loads, and more.

{% hint style="success" %}
**CLI Pro Required**: APIs require a [CLI Pro token](https://docs.slingdata.io/sling-cli/cli-pro) or [Platform Plan](https://github.com/slingdata-io/sling-docs/blob/master/concepts/sling-platform/platform.md).
{% endhint %}

## Quick Start to build a Spec

Here's a complete, working example of a Sling API spec that fetches user data from a REST API:

```yaml
name: "Example API"
description: "Simple API to get user data"

defaults:
  state:
    base_url: https://api.example.com/v1

  request:
    headers:
      Accept: "application/json"
      Authorization: "Bearer {secrets.api_token}"  # read from env.yaml secrets section

endpoints:
  users:
    description: "Retrieve list of users"
    docs: https://docs.example.com/users # docs url for endpoint
    disabled: false                      # Set to true to temporarily disable
    
    # State variables control request parameters and track values between runs
    state:
      page: 1          # Start at page 1
      limit: 100       # Fetch 100 records per request
    
    request:
      # Will resolve to https://api.example.com/v1/users
      url: '{state.base_url}/users'
      parameters:
        page: '{state.page}'
        limit: '{state.limit}'
    
    # Control how to fetch next pages
    pagination:
      next_state:
        page: '{state.page + 1}'  # Increment page number for next request
      stop_condition: "length(response.records) < state.limit"  # Stop when page isn't full
    
    # Define how to extract and process response data
    response:
      records:
        jmespath: "data.users[]"  # Extract array of users from response
        primary_key: ["id"]       # Use 'id' field to deduplicate records
```

To run this spec with Sling:

1. Save your spec somewhere (local disk, S3, SFTP, http). You can access your spec by the [location string](https://docs.slingdata.io/sling-cli/environment#location-string) convention.
2. Create a connection in your [env.yaml](https://docs.slingdata.io/sling-cli/environment#sling-env-file-envyaml) file, like this:

```yaml
connections:
  my_api:
    type: api
    spec: file:///path/to/my_api.spec.yaml  # or Github repo file, HTTP URL
    secrets:
      api_key: xxxxxxxxxxxxxxxxxx
  
  my_postgres:
    url: postgres://....
```

3. Create a replication

```yaml
source: my_api
target: my_postgres

streams:
   # '*' will read from all endpoints (endpoint name = stream name)
   # or you can simply input the specific endpoint names
  '*':
    object: my_schema.{stream_name}
    mode: full-refresh
```

4. Run Replication.

```bash
sling run -r replication.api_postgres.yaml
```

That's it!

{% hint style="info" %}
**Build Specs with AI:** You can use AI to automatically research APIs and build specs for you. See [Using AI to build API specs](https://docs.slingdata.io/sling-cli/ai#building-a-custom-api-connector) for a guided workflow.
{% endhint %}

## Official Specs

We are actively building the number of "official" Sling API Specs offered, such as:

* [Airtable](https://docs.slingdata.io/connections/api-connections/airtable)
* [HubSpot](https://docs.slingdata.io/connections/api-connections/hubspot)
* [Stripe](https://docs.slingdata.io/connections/api-connections/stripe)
* [Github](https://docs.slingdata.io/connections/api-connections/github)

Go [here](https://docs.slingdata.io/connections/api-connections) to see the full list.

Official specs don't require a full URL or Path. They are maintained by the Sling team and can be fetched by specifying the corresponding ID, such as `stripe`, `salesforce` or `github`.

```yaml
connections:
  stripe_api:
    type: api
    spec: stripe  # Use official stripe spec
    secrets:
      api_key: sk_live_xxxxxx
```

{% hint style="info" %}
If you'd like us to build a new spec, please submit a request by filling out this [Google Form](https://docs.google.com/forms/d/e/1FAIpQLScIgkpTC6C-nWaW6atc5eLFl3uUNiIakw37WL69HuPUks08aQ/viewform?usp=dialog), or by submitting a new [Github issue](https://github.com/slingdata-io/sling-cli/issues) (choosing the API Spec option).
{% endhint %}

### Forking Official Specs

When you run a connection test or use an official spec, Sling automatically downloads and caches the spec file locally at:

```
$SLING_HOME_DIR/api/specs/{spec_id}.yaml
```

For example, if you're using the `stripe` spec, after running `sling conns test my_stripe_conn`, you'll find the spec at `~/.sling/api/specs/stripe.yaml`.

To fork and customize an official spec:

1. Run a connection test to download the spec:

   ```bash
   sling conns test my_stripe_conn
   ```
2. Copy the spec from the cache folder:

   ```bash
   cp ~/.sling/api/specs/stripe.yaml ./my_custom_stripe.spec.yaml
   ```
3. Modify the spec as needed for your use case
4. Update your connection to use your custom spec:

   ```yaml
   connections:
     my_stripe:
       type: api
       spec: file://./my_custom_stripe.spec.yaml
       secrets:
         api_key: sk_live_xxxxxx
   ```

## API Spec Documentation

This section covers the details of building Sling API specifications:

* [**Structure**](https://docs.slingdata.io/concepts/api-specs/structure)**:** Understand the fundamental YAML structure of API specs, including endpoints, state management, sync variables, stream overrides, and lifecycle sequences (setup/teardown).
* [**Authentication**](https://docs.slingdata.io/concepts/api-specs/authentication)**:** Configure authentication methods including Bearer tokens, Basic Auth, OAuth2, AWS Signature V4, and custom sequence-based authentication workflows.
* [**Requests & Iteration**](https://docs.slingdata.io/concepts/api-specs/request)**:** Define HTTP requests with URLs, methods, headers, parameters, and payloads. Configure iteration to loop requests over data sets or queues, and use setup/teardown sequences for multi-step workflows.
* [**Response Processing**](https://docs.slingdata.io/concepts/api-specs/response)**:** Process API responses in multiple formats (JSON, CSV, XML, JSON Lines), extract records with JMESPath, configure deduplication strategies, and access response state for pagination and rules.
* [**Queues**](https://docs.slingdata.io/concepts/api-specs/queues)**:** Pass data between endpoints using queues for multi-step extraction workflows, such as collecting IDs from one endpoint to use in detail requests from another endpoint.
* [**Dynamic Endpoints**](https://docs.slingdata.io/concepts/api-specs/dynamic-endpoints)**:** Programmatically generate multiple endpoint configurations based on runtime data, perfect for APIs where available resources aren't known until queried.
* [**Advanced Features**](https://docs.slingdata.io/concepts/api-specs/advanced)**:** Master pagination strategies (cursor, offset, page-based), response processors for data transformation, sync state for incremental loads, and rules for error handling with intelligent retry and backoff strategies.
* [**Expression Functions**](https://docs.slingdata.io/concepts/functions)**:** Leverage built-in functions within expressions (`{...}`) for data manipulation, date operations, type casting, string operations, and control flow throughout your API spec configuration.
* [**Testing & Debugging**](https://docs.slingdata.io/concepts/api-specs/testing-debug)**:** Test your API specs using the `sling conns test` command with `--debug` and `--trace` flags to inspect request/response details, troubleshoot issues, and optimize your configuration.
* [**Troubleshooting**](https://docs.slingdata.io/concepts/api-specs/troubleshooting)**:** Common error messages, debugging techniques, and solutions for authentication, pagination, and JMESPath issues.

## API Workflow Overview

{% @mermaid/diagram content="graph TD
A\[Define API Spec YAML] --> B\[Configure Authentication]
B --> C\[Define Endpoints]
C --> D\[Configure Requests]
D --> E\[Setup Response Processing]
E --> F\[Handle Pagination]
F -->|Optional| G\[Setup Queues]
G -->|Optional| H\[Configure Additional Endpoints]
E -->|Optional| I\[Define Incremental Sync]

```
style A fill:#4a9eff,stroke:#ffffff,stroke-width:2px,color:#ffffff
style B fill:#ff8c42,stroke:#ffffff,stroke-width:2px,color:#ffffff
style C fill:#7cb342,stroke:#ffffff,stroke-width:2px,color:#ffffff
style D fill:#ffd54f,stroke:#ffffff,stroke-width:2px,color:#000000
style E fill:#26c6da,stroke:#ffffff,stroke-width:2px,color:#ffffff
style F fill:#ab47bc,stroke:#ffffff,stroke-width:2px,color:#ffffff
style G fill:#5c6bc0,stroke:#ffffff,stroke-width:2px,color:#ffffff
style H fill:#7cb342,stroke:#ffffff,stroke-width:2px,color:#ffffff
style I fill:#ab47bc,stroke:#ffffff,stroke-width:2px,color:#ffffff" %}
```

For practical examples, see the [API Examples](https://github.com/slingdata-io/sling-docs/blob/master/examples/api/README.md) section.

## Common Use Cases

| Use Case               | Key Features                                                    |
| ---------------------- | --------------------------------------------------------------- |
| Simple data extraction | Basic endpoint definition, JMESPath extraction                  |
| Paginated data         | Pagination configuration with `next_state` and `stop_condition` |
| Incremental updates    | `sync` state variables, timestamp filtering                     |
| Dependent requests     | Queues, iteration over IDs                                      |
| Authentication         | Bearer tokens, Basic auth, OAuth2                               |
| Rate limiting          | Response rules, backoff strategies                              |

> 💡 **Tip:** Start with a minimal working spec and gradually add more advanced features as needed.