Pipeline Architecture

Hookbase processes every incoming webhook through a multi-stage pipeline. Understanding this flow helps you configure sources, routes, and destinations for optimal reliability.

Processing Flow

┌─────────────────────────────────────────────────────────────┐
│                    1. INGEST                                │
│  Webhook arrives → IP Filter → Rate Limit → Event Quota    │
├─────────────────────────────────────────────────────────────┤
│                    2. VALIDATE                              │
│  Signature Verification → Schema Validation                 │
├─────────────────────────────────────────────────────────────┤
│                    3. PROCESS                               │
│  Field Encryption/Masking → Dedup Check → Store (D1 + R2)  │
├─────────────────────────────────────────────────────────────┤
│                    4. ROUTE                                 │
│  Evaluate Routes (priority order) → Filter → Transform     │
├─────────────────────────────────────────────────────────────┤
│                    5. DELIVER                               │
│  Queue → Worker → Retry → Failover → Circuit Breaker       │
│  → Notifications                                           │
└─────────────────────────────────────────────────────────────┘

Stage 1: Ingest

When a webhook arrives at https://api.hookbase.app/ingest/{orgSlug}/{sourceSlug}:

1. IP Filtering — If the source has ipFilterMode set to allowlist or denylist, the request IP is checked. Blocked requests receive 403 Forbidden.

2. Rate Limiting — If rateLimitPerMinute is configured, excess requests receive 429 Too Many Requests with a Retry-After header.

3. Event Quota — The organization's plan limits are checked. If the monthly event quota is exceeded, requests receive 429 with an upgrade prompt.

All three checks happen before any payload processing, keeping latency minimal for rejected requests.

Stage 2: Validate

1. Signature Verification — If the source has a signingSecret, the signature is validated using the scheme its provider names (or custom when it names none) (HMAC-SHA256 for GitHub, timestamp+signature for Stripe, etc.).

   • If rejectInvalidSignatures is true, invalid signatures return 401 Unauthorized
   • If false (default), the event is accepted but flagged with signatureValid: false

2. Schema Validation — If a route has a schema attached, the payload is validated against the JSON Schema. Invalid payloads result in deliveries with status schema_failed.

Stage 3: Process

1. Field Encryption — Fields matching encryptFields JSONPath expressions are encrypted using AES-256-GCM before storage. Encrypted fields are decrypted on delivery.

2. Field Masking — Fields matching maskFields expressions are masked in logs and the dashboard (e.g., user@example.com → us***@example.com). The original value is preserved for delivery.

3. Deduplication — If dedupEnabled is true, the event is checked against a sliding window. Duplicate events are silently dropped. See Deduplication Guide.

4. Storage — The event metadata is stored in D1 (SQLite) and the raw payload is stored in R2 (object storage). This separation keeps the database fast while preserving full payloads.

Stage 4: Route

Routes are evaluated in priority order (highest priority value first):

1. Route Matching — All enabled routes with the matching sourceId are selected.

2. Filter Evaluation — For each route, the filter (saved or inline) is evaluated against the event. Events that don't match are skipped for that route.

3. Transform — If the route has a transform, the payload is transformed before queuing for delivery. Each route can use a different transform for the same event.

Stage 5: Deliver

1. Queue — Deliveries are queued to Cloudflare Queues for async processing. This decouples ingestion from delivery, ensuring webhook acceptance is fast (~10ms).

2. Worker — Queue consumers pick up deliveries and POST the (optionally transformed) payload to the destination URL, including configured headers and auth.

3. Retry — Failed deliveries are retried on a fixed backoff schedule, up to 5 attempts in total:

   After attempt	Next attempt waits
   1	10 seconds
   2	30 seconds
   3	2 minutes
   4	10 minutes

   The first attempt is immediate. Any 4xx response except 429 is treated as a permanent failure and is not retried at all — including 408, so a destination that answers a timeout with 408 rather than closing the connection gets one attempt. After the fifth attempt the delivery goes to the dead-letter queue, where you can replay it.

   Info

   This schedule is not configurable. The attempt limit and the four delays are fixed in the delivery worker — there is no maxRetries, initialDelay, backoffMultiplier or maxDelay setting on a route or a destination, and no API accepts one.

4. Failover — After failoverAfterAttempts consecutive failures, deliveries are automatically routed to failover destinations. See Failover Guide.

5. Circuit Breaker — If a destination fails repeatedly, the circuit breaker trips to open state, pausing deliveries to protect the destination. See Circuit Breaker Guide.

6. Notifications — Based on route notification settings, alerts are sent via configured notification channels for failures, successes, or recovery events.

Performance Characteristics

Related

• Sources — Configure ingestion settings
• Routes — Configure routing and filtering
• Destinations — Configure delivery targets
• Deduplication — Prevent duplicate deliveries
• Circuit Breaker — Automatic failure protection
• Failover — Backup destination routing