Preview Your Transform Against Real Failed Events, Not Synthetic Samples
The transform builder always made you edit against a sample input you made up. The new Failures tab pulls the last 10 events that actually failed on a given route and runs your in-progress code against each — pass/fail status, expandable per-event output, debounced auto-rerun as you edit.
The Test/Hope/Debug Loop
The transform builder has had a sample-input box for a while: paste a JSON example, see how your transform output looks, save the transform. It works, but it has one big flaw — the sample is something you made up. It's not what your real source is actually sending.
So the loop has always been:
- Edit the transform against a fake sample.
- Save it.
- Wait for real traffic.
- Find out, eventually, that real traffic differs.
- Go back to step 1.
The transform builder now closes that loop.
A "Failures" Tab That Auto-Diffs
When you open the transform builder with a route context — either from the Edit transform & preview against this route's recent failures link on the route detail page, or by navigating to /tools/transforms/builder?id=<transformId>&routeId=<routeId> directly — a new Failures tab appears in the right sidebar.
The tab pulls the last 10 events on that route where delivery failed and runs your current in-progress transform code against each. As you edit, it re-runs them (debounced 1.5s) and the tab's passed / total badge updates live.
Each row in the tab is one failed event. Click to expand:
- Input payload — the raw event that originally failed, with redactions applied
- Transform output — what your new code produces, pretty-printed
- Original delivery error — the error from the original failed delivery, so you can see what the destination said the first time around
A passed row means your new transform parses the payload cleanly. A failed row means it still throws on that real input. That's the diff: green rows are fixed cases, red rows still need work.
Why It's an Endpoint, Not a Client Query
The backend exposes GET /api/organizations/:orgId/routes/:routeId/failed-event-samples for this. Each call returns up to 10 dedup-by-event recent failures with payloads attached (R2 or inline, with redactions applied server-side). The client then calls the existing POST /transforms/test against each sample.
We considered batching the test calls but settled on 10 sequential calls because (a) the volume is bounded, (b) it gives clear per-row status updates, and (c) it reuses the same single-test path we know is solid.
When It Matters
Sources change shapes. Provider APIs deprecate fields. Customers send you payloads with new optional fields you didn't expect. A transform that worked yesterday is one schema bump away from being broken tomorrow.
When that happens, the loop is no longer "write code → hope" — it's "write code → see the 10 most recent breakages all turn green → save." That's about as fast as schema-fix-with-confidence gets.
Pair this with the Save to route option on replay-with-edit and you have a complete workflow: edit the transform against real failed payloads, save it back to the route, and replay everything that broke — all without leaving one screen.