Your Data Export No Longer Hits a Wall at 1,000 Records

The Limit Nobody Wants to Hit

"Download My Data" is the kind of feature you build once for compliance and hope stays boring. For most accounts, ours was: request it, get a file. For accounts with a long history across several organizations, it wasn't — past a flat 1,000-record threshold, the export endpoint just returned a 400 telling you to contact support. Not a slower export. Not a partial one. A dead end, for exactly the accounts whose data mattered most to get right.

The export is now a single consolidated query for everything except payload bodies, has no upper limit on account size, and only asks you to wait when there's real work to do.

What's New

• One query instead of a fan-out — a single database function now does the full multi-table join and aggregation for a user's export in one round trip, replacing dozens of separate per-organization queries
• No more hard ceiling — the old flat "over 1,000 records, request denied" cutoff is gone entirely; large accounts still export, they just take a bit longer
• Instant exports stay instant — accounts under the new payload threshold get their file back synchronously, same as before
• A real background path for everything else — large exports run as a queued job that hydrates payload bodies in checkpointed batches, with a live progress bar in the dashboard and an automatic download when it's ready
• 24-hour caching — a completed export is cached and served instantly on repeat requests instead of being regenerated from scratch every time

The Ceiling Was Never About Rows

The old 1,000-record cutoff looked like a row-count limit, but the actual constraint underneath it was how many outbound calls a single request could make. Every event in an export carries a payload body that lives in separate storage, not inline with the rest of the record — so building the export meant one storage fetch per payload on top of the metadata queries, and a single request is capped at a fixed number of those. Past a certain export size, the payload hydration alone would blow that budget before the request could even finish. The 1,000-record cutoff wasn't a policy decision, it was where the code happened to run out of budget.

Fixing that meant separating two different costs that had been bundled into one request: pulling the metadata (cheap, one consolidated query, no real ceiling) from hydrating the payload bodies (the actually expensive part, one fetch at a time). A threshold now draws the line for what still happens synchronously — under that many payloads to hydrate, you get your file back in the same request, same as always. Over it, the export becomes a background job instead of a rejection.

What the Background Path Actually Does

A new background worker picks up large exports and hydrates their payloads in checkpointed batches, re-queuing itself after each batch until the whole export is done. There's no size ceiling on this path — a batch finishes, the worker checkpoints its progress and re-queues the rest, and it keeps going for as long as it takes. The per-table row limits that used to cap out in the low thousands are now far higher, because without inlined payload bodies those rows are cheap — metadata and a storage reference, not the data itself.

On the dashboard side, the Download My Data card now branches on what the export endpoint actually returns: a file directly (small account, or a cached export from the last 24 hours), or an in-progress job that's still hydrating. In the second case, you get a live progress bar instead of silence, and the file downloads automatically the moment the job completes — no second click required to notice it's done.

One Less Thing to Keep in Sync

Completed exports are cached for 24 hours and served straight from cache on a repeat request instead of regenerating the whole thing. That cache also made a separate piece of infrastructure redundant: the export endpoint used to have its own dedicated rate limiter to keep someone from re-triggering an expensive export on every click. With a real completed-export cache doing that job as a side effect, the old limiter was just duplicate state tracking the same thing two ways — it's gone, and a retention sweep now cleans up expired or stuck export jobs and their cached files, since none of it expires on its own.

Try It

Open Settings → Download My Data. Nothing about the button changed — what changed is what happens after you click it, whether that's an instant file or a progress bar that finishes on its own.