
When we promised fifteen-minute syncs across forty integrations, our original pipeline could not keep up. Large backfills blocked fresh updates, and a single slow API could delay everything behind it.
What changed
Each source now runs in its own queue with independent retries and rate limits. Backfills run at low priority, so new data always arrives first.
The result
Median sync time fell from nine minutes to under two, and failed syncs now recover on their own in most cases.