Company

·

8 min read

Why we rebuilt our sync engine from scratch

Fifteen-minute syncs across 40 integrations meant rethinking how we queue, retry and backfill. Notes from the team.

Cover image for “Why we rebuilt our sync engine from scratch”

When we promised fifteen-minute syncs across forty integrations, our original pipeline could not keep up. Large backfills blocked fresh updates, and a single slow API could delay everything behind it.

What changed

Each source now runs in its own queue with independent retries and rate limits. Backfills run at low priority, so new data always arrives first.

The result

Median sync time fell from nine minutes to under two, and failed syncs now recover on their own in most cases.

Newsletter

One useful metric, every other Tuesday.

Short, practical notes on revenue, retention and forecasting. No spam, and you can unsubscribe in one click.

Create a free website with Framer, the website builder loved by startups, designers and agencies.