# Http HTTP hooks enable you to make HTTP requests to external services as part of your replication workflow. This is particularly useful for integrating with external APIs, sending notifications, or triggering other systems. ## Configuration ```yaml - type: http url: "https://api.example.com/webhook" # Required method: GET # Optional: GET/POST/PUT/DELETE (default: GET) payload: '{"status": "{run.status}"}' # Optional: Request body (also accepts YAML object) timeout: 10 # Optional: Request timeout seconds (default is 30sec) headers: # Optional: Request headers Authorization: "Bearer token" auth: # Optional: Authentication configuration type: basic # basic, aws-sigv4, or hmac username: "myuser" password: "mypass" proxy: "http://user:pass@proxy.example.com:8080" # Optional: HTTP/HTTPS proxy URL write_to: local/path/to/response.json # Optional: Save response to a file on_failure: abort # Optional: abort/warn/quiet/skip id: my_id # Optional. Will be generated. Use `log` hook with {runtime_state} to view state. ``` ## Properties | Property | Required | Description | | ----------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | | url | Yes | The URL to send the request to | | method | No | HTTP method (GET/POST/PUT/DELETE). Defaults to GET | | payload | No | The request body (for POST/PUT requests) | | headers | No | Map of HTTP headers to include in the request | | auth | No | Authentication configuration (see [Authentication](#authentication) section) | | proxy | No | HTTP/HTTPS proxy URL (e.g., `http://user:pass@proxy.example.com:8080`) | | write\_to | No | [Location](/sling-cli/environment.md#location-string) to save the response bytes to a file (e.g., `local/path/to/response.json`, `s3/folder/response.csv`) | | on\_failure | No | What to do if the request fails (abort/warn/quiet/skip) | ## Output When the HTTP hook executes successfully, it returns the following output that can be accessed in subsequent hooks: ```yaml status: success # Status of the hook execution request: # Details of the request made method: "GET" # HTTP method used url: "https://api.example.com/webhook" # The URL called headers: # Headers sent with the request Authorization: "Bearer token" payload: '{"status": "success"}' # The request body sent response: # Details of the response received headers: # Response headers Content-Type: "application/json" # ... other response headers status: "200 OK" # HTTP status message status_code: 200 # HTTP status code size: 123456 # Response bytes size text: "Response body as text" # Raw response body json: # Parsed JSON response (if response is JSON) key: "value" # ... rest of JSON structure written_to: # Details of saved response file (if write_to is specified) location: "local/path/to/response.json" # The write_to location uri: "file:///absolute/path/to/response.json" # Absolute URI of saved file bytes_written: 1234 # Number of bytes written to file ``` You can access these values in subsequent hooks using the following syntax (`jmespath`): * `{state.hook_id.status}` - Status of the hook execution * `{state.hook_id.request.method}` - HTTP method used * `{state.hook_id.request.url}` - The URL called * `{state.hook_id.request.headers}` - Request headers * `{state.hook_id.request.payload}` - Request body * `{state.hook_id.response.status}` - HTTP status message * `{state.hook_id.response.status_code}` - HTTP status code * `{state.hook_id.response.text}` - Raw response body * `{state.hook_id.response.json}` - Parsed JSON response * `{state.hook_id.response.size}` - Size of response in bytes * `{state.hook_id.written_to.location}` - Location where response was saved * `{state.hook_id.written_to.uri}` - Absolute URI of saved response file ## Examples ### Slack Notification Send a notification to Slack after replication completes: ```yaml hooks: post: - type: http url: "https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK" method: POST payload: | { "text": "Replication Status Update", "blocks": [ { "type": "section", "text": { "type": "mrkdwn", "text": "*Stream:* {run.stream.name}\n*Status:* {run.status}\n*Rows Processed:* {run.total_rows}\n*Duration:* {run.duration} seconds" } } ] } on_failure: warn ``` ### Microsoft Teams Alert Send an alert to Microsoft Teams when replication fails: ```yaml hooks: post: - type: http if: run.status == "error" url: "https://your-teams-webhook-url" method: POST payload: | { "@type": "MessageCard", "@context": "http://schema.org/extensions", "themeColor": "FF0000", "summary": "Replication Failed", "sections": [{ "activityTitle": "⚠️ Replication Failed", "facts": [ { "name": "Stream", "value": "{run.stream.name}" }, { "name": "Target Table", "value": "{run.object.full_name}" }, { "name": "Start Time", "value": "{run.start_time}" }, { "name": "End Time", "value": "{run.end_time}" } ] }] } ``` ### REST API Integration Fetch configuration from an external API before starting replication: ```yaml hooks: pre: - type: http url: "https://api.company.com/v1/config/{run.stream.name}" method: GET headers: Authorization: "Bearer {source.api_key}" Content-Type: "application/json" ``` ### API with Basic Authentication Call an API endpoint that requires basic authentication: ```yaml hooks: post: - type: http url: "https://api.example.com/v1/notify" method: POST auth: type: basic username: "{env.API_USER}" password: "{env.API_PASS}" payload: | { "event": "replication_complete", "stream": "{run.stream.name}", "rows": {run.total_rows} } ``` ### API with HMAC Signature Call an API that requires HMAC signature authentication: ```yaml hooks: post: - type: http url: "https://partner-api.example.com/webhooks/events" method: POST auth: type: hmac algorithm: sha256 secret: "{env.PARTNER_HMAC_SECRET}" signing_string: "{http_method}\n{http_path}\n{unix_time}\n{http_body_sha256}" request_headers: X-Signature: "sha256={signature}" X-Timestamp: "{unix_time}" payload: | { "event_type": "replication.complete", "stream": "{run.stream.name}", "status": "{run.status}", "rows_processed": {run.total_rows}, "timestamp": "{run.end_time}" } ``` ### Trigger External Workflow Trigger an external workflow system after successful replication: ```yaml hooks: post: - type: http if: run.status == "success" url: "https://api.workflow-system.com/v1/triggers" method: POST headers: Authorization: "ApiKey {target.api_key}" payload: | { "workflow_id": "data-quality-check", "parameters": { "table_name": "{run.object.full_name}", "row_count": {run.total_rows}, "execution_date": "{timestamp.date}" } } ``` ### Data Quality Service Integration Send data quality metrics to a monitoring service: ```yaml hooks: post: - type: http url: "https://metrics-api.company.com/v1/metrics" method: POST headers: X-API-Key: "{target.metrics_api_key}" payload: | { "metric_type": "data_quality", "timestamp": "{run.end_time}", "metrics": { "table_name": "{run.object.full_name}", "record_count": {run.total_rows}, "processing_time_seconds": {run.duration}, "bytes_processed": {run.total_bytes} }, "tags": { "environment": "{env.environment}", "stream": "{run.stream.name}" } } ``` ### Error Tracking Integration Send error details to an error tracking service when replication fails: ```yaml hooks: post: - type: http if: run.status == "error" url: "https://api.errortrackers.com/v1/errors" method: POST headers: Authorization: "Bearer {target.error_tracking_token}" payload: | { "error": { "name": "Replication Failed", "environment": "{target.environment}", "metadata": { "stream": "{run.stream.name}", "target_table": "{run.object.full_name}", "start_time": "{run.start_time}", "end_time": "{run.end_time}", "rows_processed": {run.total_rows} } } } on_failure: warn ``` ### Download and Save API Response Fetch data from an API and save the response to a file: ```yaml hooks: pre: - type: http id: fetch_config url: "https://api.example.com/v1/config" method: GET headers: Authorization: "Bearer {env.API_TOKEN}" write_to: local/configs/api_response.json # Use the saved response location in a subsequent hook - type: log message: "API response saved to {state.fetch_config.written_to.uri} ({state.fetch_config.written_to.bytes_written} bytes)" ``` ## Authentication The HTTP hook supports multiple authentication methods through the `auth` property: ### Basic Authentication Use HTTP Basic Authentication with username and password: ```yaml - type: http url: "https://api.example.com/endpoint" auth: type: basic username: "{env.API_USERNAME}" password: "{env.API_PASSWORD}" ``` ### AWS Signature V4 Sign requests using AWS Signature Version 4 (for AWS services): ```yaml - type: http url: "https://my-service.us-east-1.amazonaws.com/endpoint" auth: type: aws-sigv4 aws_service: execute-api aws_region: us-east-1 aws_access_key_id: "{env.AWS_ACCESS_KEY_ID}" aws_secret_access_key: "{env.AWS_SECRET_ACCESS_KEY}" # Optional: # aws_session_token: "{env.AWS_SESSION_TOKEN}" # aws_profile: "my-profile" ``` **Note**: AWS credentials can also be loaded from environment variables or AWS profiles if not explicitly provided. ### HMAC Signature Sign requests using HMAC (Hash-based Message Authentication Code) for custom API authentication: ```yaml - type: http url: "https://api.example.com/endpoint" auth: type: hmac algorithm: sha256 # or sha512 (default: sha256) secret: "{env.HMAC_SECRET}" signing_string: "{http_method}\n{http_path}\n{unix_time}\n{http_body_sha256}" request_headers: X-Signature: "{signature}" X-Timestamp: "{unix_time}" # Optional nonce: # X-Nonce: "{nonce}" # nonce_length: 16 # Optional: generate random nonce (in bytes) ``` **Available template variables for `signing_string` and `request_headers`:** * `{http_method}` - HTTP method (GET, POST, etc.) * `{http_path}` - Request path including query string * `{http_body_md5}` - MD5 hash of request body (hex) * `{http_body_sha1}` - SHA1 hash of request body (hex) * `{http_body_sha256}` - SHA256 hash of request body (hex) * `{http_body_sha512}` - SHA512 hash of request body (hex) * `{http_body_raw}` - Raw request body as string * `{http_query}` - Canonical query string (sorted, URL-encoded) * `{http_headers}` - Canonical headers string (lowercase, sorted) * `{unix_time}` - Current Unix timestamp (seconds) * `{unix_time_ms}` - Current Unix timestamp (milliseconds) * `{date_iso}` - Current date in ISO 8601 format (RFC3339) * `{date_rfc1123}` - Current date in RFC1123 format * `{nonce}` - Random nonce (if `nonce_length` is set) * `{signature}` - The computed HMAC signature (available in `request_headers`) **Example with complete HMAC flow:** ```yaml - type: http url: "https://api.partner.com/v1/orders" method: POST payload: '{"order_id": "12345"}' auth: type: hmac algorithm: sha256 secret: "{env.PARTNER_API_SECRET}" signing_string: "{http_method}\n{http_path}\n{unix_time}\n{http_body_sha256}" nonce_length: 16 request_headers: Authorization: "HMAC-SHA256 {signature}" X-Timestamp: "{unix_time}" X-Nonce: "{nonce}" ``` ### Bearer Token (via Headers) For Bearer token authentication, use the `headers` property instead of `auth`: ```yaml - type: http url: "https://api.example.com/endpoint" headers: Authorization: "Bearer {env.API_TOKEN}" ``` ## Proxy Configuration If your network requires routing HTTP requests through a proxy server, you can configure it using the `proxy` property: ```yaml - type: http url: "https://api.example.com/endpoint" proxy: "http://proxy.company.com:8080" ``` ### Proxy with Authentication For proxies that require authentication, include credentials in the proxy URL: ```yaml - type: http url: "https://api.example.com/endpoint" proxy: "http://{env.PROXY_USER}:{env.PROXY_PASS}@proxy.company.com:8080" ``` The proxy URL format is: `http://[username:password@]host:port` **Notes:** * Both HTTP and HTTPS proxies are supported * Authentication credentials in the proxy URL are automatically handled * The proxy applies to all HTTP/HTTPS requests made by this hook --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.slingdata.io/concepts/hooks/http.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.