# API Specs Sling API specs are YAML files that define how to interact with REST APIs. They provide a structured way to specify authentication methods, define endpoints, configure request parameters, handle pagination, process responses, manage state for incremental loads, and more. {% hint style="success" %} **CLI Pro Required**: APIs require a [CLI Pro token](https://docs.slingdata.io/sling-cli/cli-pro) or [Platform Plan](https://github.com/slingdata-io/sling-docs/blob/master/concepts/sling-platform/platform.md). {% endhint %} ## Quick Start to build a Spec Here's a complete, working example of a Sling API spec that fetches user data from a REST API: ```yaml name: "Example API" description: "Simple API to get user data" defaults: state: base_url: https://api.example.com/v1 request: headers: Accept: "application/json" Authorization: "Bearer {secrets.api_token}" # read from env.yaml secrets section endpoints: users: description: "Retrieve list of users" docs: https://docs.example.com/users # docs url for endpoint disabled: false # Set to true to temporarily disable # State variables control request parameters and track values between runs state: page: 1 # Start at page 1 limit: 100 # Fetch 100 records per request request: # Will resolve to https://api.example.com/v1/users url: '{state.base_url}/users' parameters: page: '{state.page}' limit: '{state.limit}' # Control how to fetch next pages pagination: next_state: page: '{state.page + 1}' # Increment page number for next request stop_condition: "length(response.records) < state.limit" # Stop when page isn't full # Define how to extract and process response data response: records: jmespath: "data.users[]" # Extract array of users from response primary_key: ["id"] # Use 'id' field to deduplicate records ``` To run this spec with Sling: 1. Save your spec somewhere (local disk, S3, SFTP, http). You can access your spec by the [location string](https://docs.slingdata.io/sling-cli/environment#location-string) convention. 2. Create a connection in your [env.yaml](https://docs.slingdata.io/sling-cli/environment#sling-env-file-envyaml) file, like this: ```yaml connections: my_api: type: api spec: file:///path/to/my_api.spec.yaml # or Github repo file, HTTP URL secrets: api_key: xxxxxxxxxxxxxxxxxx my_postgres: url: postgres://.... ``` 3. Create a replication ```yaml source: my_api target: my_postgres streams: # '*' will read from all endpoints (endpoint name = stream name) # or you can simply input the specific endpoint names '*': object: my_schema.{stream_name} mode: full-refresh ``` 4. Run Replication. ```bash sling run -r replication.api_postgres.yaml ``` That's it! {% hint style="info" %} **Build Specs with AI:** You can use AI to automatically research APIs and build specs for you. See [Using AI to build API specs](https://docs.slingdata.io/sling-cli/ai#building-a-custom-api-connector) for a guided workflow. {% endhint %} ## Official Specs We are actively building the number of "official" Sling API Specs offered, such as: * [Airtable](https://docs.slingdata.io/connections/api-connections/airtable) * [HubSpot](https://docs.slingdata.io/connections/api-connections/hubspot) * [Stripe](https://docs.slingdata.io/connections/api-connections/stripe) * [Github](https://docs.slingdata.io/connections/api-connections/github) Go [here](https://docs.slingdata.io/connections/api-connections) to see the full list. Official specs don't require a full URL or Path. They are maintained by the Sling team and can be fetched by specifying the corresponding ID, such as `stripe`, `salesforce` or `github`. ```yaml connections: stripe_api: type: api spec: stripe # Use official stripe spec secrets: api_key: sk_live_xxxxxx ``` {% hint style="info" %} If you'd like us to build a new spec, please submit a request by filling out this [Google Form](https://docs.google.com/forms/d/e/1FAIpQLScIgkpTC6C-nWaW6atc5eLFl3uUNiIakw37WL69HuPUks08aQ/viewform?usp=dialog), or by submitting a new [Github issue](https://github.com/slingdata-io/sling-cli/issues) (choosing the API Spec option). {% endhint %} ### Forking Official Specs When you run a connection test or use an official spec, Sling automatically downloads and caches the spec file locally at: ``` $SLING_HOME_DIR/api/specs/{spec_id}.yaml ``` For example, if you're using the `stripe` spec, after running `sling conns test my_stripe_conn`, you'll find the spec at `~/.sling/api/specs/stripe.yaml`. To fork and customize an official spec: 1. Run a connection test to download the spec: ```bash sling conns test my_stripe_conn ``` 2. Copy the spec from the cache folder: ```bash cp ~/.sling/api/specs/stripe.yaml ./my_custom_stripe.spec.yaml ``` 3. Modify the spec as needed for your use case 4. Update your connection to use your custom spec: ```yaml connections: my_stripe: type: api spec: file://./my_custom_stripe.spec.yaml secrets: api_key: sk_live_xxxxxx ``` ## API Spec Documentation This section covers the details of building Sling API specifications: * [**Structure**](https://docs.slingdata.io/concepts/api-specs/structure)**:** Understand the fundamental YAML structure of API specs, including endpoints, state management, sync variables, stream overrides, and lifecycle sequences (setup/teardown). * [**Authentication**](https://docs.slingdata.io/concepts/api-specs/authentication)**:** Configure authentication methods including Bearer tokens, Basic Auth, OAuth2, AWS Signature V4, and custom sequence-based authentication workflows. * [**Requests & Iteration**](https://docs.slingdata.io/concepts/api-specs/request)**:** Define HTTP requests with URLs, methods, headers, parameters, and payloads. Configure iteration to loop requests over data sets or queues, and use setup/teardown sequences for multi-step workflows. * [**Response Processing**](https://docs.slingdata.io/concepts/api-specs/response)**:** Process API responses in multiple formats (JSON, CSV, XML, JSON Lines), extract records with JMESPath, configure deduplication strategies, and access response state for pagination and rules. * [**Queues**](https://docs.slingdata.io/concepts/api-specs/queues)**:** Pass data between endpoints using queues for multi-step extraction workflows, such as collecting IDs from one endpoint to use in detail requests from another endpoint. * [**Dynamic Endpoints**](https://docs.slingdata.io/concepts/api-specs/dynamic-endpoints)**:** Programmatically generate multiple endpoint configurations based on runtime data, perfect for APIs where available resources aren't known until queried. * [**Advanced Features**](https://docs.slingdata.io/concepts/api-specs/advanced)**:** Master pagination strategies (cursor, offset, page-based), response processors for data transformation, sync state for incremental loads, and rules for error handling with intelligent retry and backoff strategies. * [**Expression Functions**](https://docs.slingdata.io/concepts/functions)**:** Leverage built-in functions within expressions (`{...}`) for data manipulation, date operations, type casting, string operations, and control flow throughout your API spec configuration. * [**Testing & Debugging**](https://docs.slingdata.io/concepts/api-specs/testing-debug)**:** Test your API specs using the `sling conns test` command with `--debug` and `--trace` flags to inspect request/response details, troubleshoot issues, and optimize your configuration. * [**Troubleshooting**](https://docs.slingdata.io/concepts/api-specs/troubleshooting)**:** Common error messages, debugging techniques, and solutions for authentication, pagination, and JMESPath issues. ## API Workflow Overview {% @mermaid/diagram content="graph TD A\[Define API Spec YAML] --> B\[Configure Authentication] B --> C\[Define Endpoints] C --> D\[Configure Requests] D --> E\[Setup Response Processing] E --> F\[Handle Pagination] F -->|Optional| G\[Setup Queues] G -->|Optional| H\[Configure Additional Endpoints] E -->|Optional| I\[Define Incremental Sync] ``` style A fill:#4a9eff,stroke:#ffffff,stroke-width:2px,color:#ffffff style B fill:#ff8c42,stroke:#ffffff,stroke-width:2px,color:#ffffff style C fill:#7cb342,stroke:#ffffff,stroke-width:2px,color:#ffffff style D fill:#ffd54f,stroke:#ffffff,stroke-width:2px,color:#000000 style E fill:#26c6da,stroke:#ffffff,stroke-width:2px,color:#ffffff style F fill:#ab47bc,stroke:#ffffff,stroke-width:2px,color:#ffffff style G fill:#5c6bc0,stroke:#ffffff,stroke-width:2px,color:#ffffff style H fill:#7cb342,stroke:#ffffff,stroke-width:2px,color:#ffffff style I fill:#ab47bc,stroke:#ffffff,stroke-width:2px,color:#ffffff" %} ``` For practical examples, see the [API Examples](https://github.com/slingdata-io/sling-docs/blob/master/examples/api/README.md) section. ## Common Use Cases | Use Case | Key Features | | ---------------------- | --------------------------------------------------------------- | | Simple data extraction | Basic endpoint definition, JMESPath extraction | | Paginated data | Pagination configuration with `next_state` and `stop_condition` | | Incremental updates | `sync` state variables, timestamp filtering | | Dependent requests | Queues, iteration over IDs | | Authentication | Bearer tokens, Basic auth, OAuth2 | | Rate limiting | Response rules, backoff strategies | > 💡 **Tip:** Start with a minimal working spec and gradually add more advanced features as needed. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.slingdata.io/concepts/api-specs.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.