Why Webhooks Arrive Out of Order (and How to Handle It)

Webhooks Are Not a Stream

It is tempting to think of an incoming webhook feed like a tail on a log file: events show up in the order they happened, one after another. They do not. A webhook feed is a pile of independent HTTP requests racing each other to your endpoint, and the order they land in has almost nothing to do with the order the events occurred.

The classic symptom looks like a bug in your code:

received  10:00:04.120   order.updated   status=shipped
received  10:00:04.180   order.created   status=pending

The order.updated arrived before the order.created. Your handler tries to update a row that does not exist yet, or worse, it inserts a "shipped" order and then the late created event overwrites it back to "pending." Now your database says an already-shipped order is waiting to be picked.

This is not an edge case you can engineer away by asking the provider nicely. It is inherent to how webhooks work. The durable fix is to design handlers that do not care about arrival order at all.

Why Order Falls Apart

There are several independent reasons events show up scrambled, and they compound.

• Retries reorder relative to fresh events. When a delivery fails, it gets retried seconds or minutes later. Meanwhile newer events for the same resource keep flowing. A retried order.created can easily land after the order.updated that logically follows it.
• Senders deliver in parallel. Most providers use worker pools, not a single ordered queue per subscriber. Two events emitted milliseconds apart go out on different workers and finish in whatever order the network decides.
• Multiple producers. The created event might come from the checkout service and the updated from the fulfillment service. Those are different systems with different queues and different latencies. There is no shared clock ordering them.
• Network variance. Two requests over the public internet do not arrive in send order. TLS handshakes, routing, and retries at the transport layer all add jitter.
• Clock skew. Even the timestamps inside the payloads can disagree, because they were stamped by different machines whose clocks drift. You cannot fully trust created_at from one service against created_at from another.

Put these together and the honest position is simple: webhooks are fundamentally unordered, and no amount of provider configuration changes that. Design for it.

Strategy 1: Treat Each Event as a Fact About a Point in Time

Stop thinking "process events in sequence." Start thinking "each event tells me something that was true at a specific moment; my job is to fold it into current state correctly."

An order.updated with status=shipped and version=7 is a fact: at version 7, this order was shipped. It does not matter whether it arrives first, last, or twice. If your handler knows how to apply that fact idempotently and by version, arrival order becomes irrelevant.

That reframing drives every technique below.

Strategy 2: Make Handlers Idempotent, Then Dedup Exact Retries

Order-independence is worthless if replaying the same event corrupts state. So the foundation is idempotency: running the same event once or ten times must leave the system in the same place. We covered the mechanics in depth in idempotency keys for webhooks — use the provider's event id as your key, record processed ids, and short-circuit anything you have already applied.

On top of that, drop the exact retries that carry no new information. Hookbase does this for you: deduplication drops exact duplicate deliveries of the same event before they ever reach your handler, so a sender that fires the same delivery three times only costs you one unit of work. (More on the tradeoffs in webhook deduplication.)

Idempotency plus dedup removes the duplicate half of the problem. It does not fix stale overwrites — for that you need a version guard.

Strategy 3: Order by the Payload's Own Field, Not by Arrival

This is the technique that actually solves out-of-order updates. Do not use "when did this arrive." Use a monotonic field the sender put inside the payload: a version, a sequence, a revision counter, or a trustworthy updated_at.

Then make every write last-writer-wins by that field. A stale event that arrives late simply loses.

UPDATE orders
SET status         = $2,
    version        = $3,
    updated_at     = $4
WHERE id           = $1
  AND version      < $3;   -- only apply if incoming is newer

If the late order.created (version 3) shows up after the order.updated (version 7), the WHERE version < 3 never matches the already-stored version 7, so nothing happens. The newer state survives.

For upserts, guard the conflict the same way:

INSERT INTO orders (id, status, version, updated_at)
VALUES ($1, $2, $3, $4)
ON CONFLICT (id) DO UPDATE
SET status     = EXCLUDED.status,
    version    = EXCLUDED.version,
    updated_at = EXCLUDED.updated_at
WHERE EXCLUDED.version > orders.version;

Now it does not matter which event lands first. The row always converges to the highest version you have seen. This is the single highest-leverage change most teams can make.

A version guard only works if the field is genuinely monotonic per resource. A wall-clock updated_at from a single source usually qualifies; a timestamp assembled from several skewed clocks does not. Prefer an integer sequence the provider guarantees increases.

Strategy 4: Reconcile by Re-Fetching Current State

Sometimes the payload does not carry a reliable version, or you simply do not want to trust a body that may already be stale by the time you read it. The answer is fetch-on-notify: treat the webhook as a doorbell, not a package. When it rings, call the provider's API and read the current state yourself.

async function onWebhook(event) {
  // Don't trust event.data — it may be stale or out of order.
  const order = await provider.orders.retrieve(event.data.id);
  await upsertOrder(order); // authoritative, current state
}

Because you read the latest state at processing time, ordering between webhooks stops mattering. A late notification just triggers another fetch that returns the same current state. This "thin payload" pattern pairs naturally with the reliability tradeoffs in webhooks vs polling — you get push latency with pull correctness. The cost is an extra API call per event and a dependency on the provider's read endpoint being up.

Strategy 5: The Child-Before-Parent Case

The hardest variant is a child event that references a parent you have not seen yet: an invoice.line_item.created before the invoice.created, or a comment before its issue. A version guard on the child does not help, because the parent row simply is not there.

You have three reasonable options, roughly in order of preference:

• Fetch the parent. Same fetch-on-notify idea. If the child references a parent id you do not have, retrieve the parent from the API and materialize it first, then apply the child.
• Tolerate the gap. Upsert a stub parent row from the foreign key, mark it incomplete, and let the real parent.created fill it in when it arrives. Your version guard keeps the eventual real data from being clobbered.
• Buffer briefly. If neither fits, hold the orphan child for a few seconds and retry locally. Keep the window short and always fall back to fetch or stub — never block a handler indefinitely waiting for an event that may never come.

Avoid the temptation to reject the child with a 5xx and hope the provider's retry lands after the parent. That works sometimes and turns your ordering problem into a retry-storm problem the rest of the time.

Where Hookbase Fits

Hookbase stamps every event with a stable event id and a received-at timestamp you can see in the dashboard, and it dedups exact retries so replayed deliveries do not do double work. That kills the duplicate problem and gives you a reliable key for idempotency.

What it deliberately does not do is promise strict global ordering of delivery — because no relay honestly can. The pipeline (sources to routes to filters to transforms to destinations, with automatic retries) is built to deliver reliably, and reliable delivery means retries, and retries mean reordering.

So use the event id and dedup for the duplicate half, and own the ordering half in your handlers: version-guard your writes, fetch current state when in doubt, and treat every event as a timestamped fact rather than a step in a sequence. Do that once and the out-of-order bug reports stop for good — regardless of which provider you add next. Want to watch reordering happen before it hits production? Fire a few events at the webhook simulator and see how your handler holds up.