Skip to main content
The Webhook destination delivers structured pipeline output to an HTTP endpoint. It supports configurable batching, retry with exponential backoff, and idempotency keys for safe retries.

​
Configuration

destination:
  type: "webhook"
  url: "https://api.example.com/events"
  method: "POST"
  headers:
    Authorization: "Bearer ${WEBHOOK_TOKEN}"
    Content-Type: "application/json"
  batch_size: 1
  timeout_ms: 10000
  retry:
    max_attempts: 3
    backoff_ms: 1000

​
Fields

FieldTypeRequiredDefaultDescription
urlstringyes—Webhook endpoint URL
methodstringnoPOSTHTTP method
headersmapno—Custom HTTP headers
batch_sizeintno1Records per request
timeout_mslongno10,000Request timeout (ms)
retry.max_attemptsintno3Max retry attempts
retry.backoff_mslongno1,000Initial backoff delay (ms)

​
Payload Format

The request body format depends on the batch_size setting.

​
Single-record payload (batch_size = 1)

{
  "project": "analytics-prod",
  "stream": "hl_fills",
  "record": { "..." }
}

​
Batched payload (batch_size > 1)

{
  "project": "analytics-prod",
  "stream": "hl_fills",
  "records": [ { "..." }, { "..." } ],
  "count": 2
}

​
Idempotency

Every request includes an X-Idempotency-Key header derived from source metadata, allowing your endpoint to safely deduplicate retried deliveries.
ModeKey Format
Single record{topic}-{partition}-{offset}
Batch{topic}-{partition}-{minOffset}_{maxOffset}

​
Retry Behavior

The destination retries failed requests using exponential backoff. The initial delay is set by retry.backoff_ms and doubles on each subsequent attempt, up to retry.max_attempts.
ResponseAction
2xxSuccess - record acknowledged
429Retry, honoring Retry-After header if present; otherwise use backoff
5xxRetry with exponential backoff
4xx (not 429)Permanent failure - batch is skipped