01 · Foundations
Why APIs enforce rate limits
Rate limits turn shared capacity into a predictable contract instead of a race for resources.
An API has finite compute, database connections, and downstream capacity. A rate limit defines how many requests a client may make during a window so that one integration cannot exhaust those shared resources. The limit protects availability, controls cost, and gives every client a more predictable experience.
A well-behaved client treats the limit as part of the API contract. It observes response headers, slows down when capacity is low, and retries only when the server indicates that another attempt is reasonable. Retrying immediately and repeatedly creates more load at the exact moment the service is asking for less.
02 · Response signals
Reading the headers
Headers expose the size, remaining capacity, and reset point of the current request window.
Header names vary by provider, but most communicate the same three facts: the total allowance, the requests remaining, and the time when the budget resets. Read the provider documentation before assuming whether a reset value is a duration, a Unix timestamp, or a formatted date.
| Header | What it communicates | Client response |
|---|---|---|
RateLimit-Limit | Maximum requests allowed in the current policy window | Use it to pace planned work |
RateLimit-Remaining | Requests still available before the limit is reached | Reduce concurrency as it approaches zero |
RateLimit-Reset | When the current budget becomes available again | Schedule later work after the reset |
Retry-After | How long to wait before retrying this response | Honor it before calculating a fallback delay |
HTTP/1.1 200 OK
RateLimit-Limit: 100
RateLimit-Remaining: 7
RateLimit-Reset: 4203 · Failure handling
When the server returns 429
A 429 response is an instruction to pause, not proof that the request can never succeed.
The HTTP status 429 Too Many Requests means the client has exceeded a rate policy. The response may include Retry-After as either a number of seconds or an HTTP date. A client should prefer that server-provided instruction because the server understands its own recovery window better than the caller does.
HTTP/1.1 429 Too Many Requests
Content-Type: application/json
Retry-After: 4
{ "error": "rate_limit_exceeded" }04 · Retry strategy
Exponential backoff
Each unsuccessful attempt increases the delay, creating room for a constrained service to recover.
With exponential backoff, the client multiplies a base delay after each unsuccessful attempt. A simple schedule might wait 500 ms, 1 second, 2 seconds, and 4 seconds. The growing interval reduces synchronized pressure and prevents a tight retry loop.
Production clients also need boundaries: a maximum attempt count, a maximum delay, a request timeout, and cancellation support. If Retry-After is present, treat it as the minimum server-directed wait; otherwise calculate a bounded backoff delay.
05 · Traffic shaping
Why backoff needs jitter
Random variation keeps many clients from retrying in the same synchronized wave.
If thousands of clients fail together and use identical delays, they will wake together and create another traffic spike. This is sometimes called the thundering-herd problem. Jitter adds bounded randomness so retry attempts spread across the recovery window.
Visual reference
One possible retry timeline
- 1Initial request0 msReceives 429
- 2Retry 1620 ms500 ms base + jitter
- 3Retry 21.3 s1 s base + jitter
- 4Retry 32.2 s2 s base + jitter
06 · Implementation
A bounded TypeScript client
Combine server guidance, bounded attempts, cancellation, exponential delay, and jitter.
This compact example retries only 429 responses, respects Retry-After when it contains seconds, and otherwise uses full jitter. A production implementation should also classify network failures, parse HTTP-date values, record observability data, and follow the provider's exact policy.
const sleep = (ms: number, signal?: AbortSignal) =>
new Promise<void>((resolve, reject) => {
const timer = setTimeout(resolve, ms);
signal?.addEventListener("abort", () => {
clearTimeout(timer);
reject(signal.reason);
}, { once: true });
});
export async function requestWithBackoff(
url: string,
signal?: AbortSignal,
) {
const maxAttempts = 4;
for (let attempt = 0; attempt < maxAttempts; attempt += 1) {
const response = await fetch(url, { signal });
if (response.status !== 429) return response;
if (attempt === maxAttempts - 1) break;
const retryAfter = Number(response.headers.get("Retry-After"));
const ceiling = 500 * 2 ** attempt;
const delay = Number.isFinite(retryAfter)
? retryAfter * 1_000
: Math.random() * ceiling;
await sleep(Math.min(delay, 8_000), signal);
}
throw new Error("Request remained rate limited");
}