LLM API 429 Errors: The Three Causes and How to Fix Each

HTTP 429 is the least informative error an LLM API returns, because three genuinely different failures share the same status code — and each one has a different fix. Retry code fixes exactly one of them. For the second, retrying just delays the inevitable, and for the third, retrying is pure waste. This post is the deep dive behind the error-code quick reference: how to tell the three apart, the production-grade retry loop for the one case where retrying helps, and the structural change that stops 429s from happening in the first place.

One status code, three different failures

Misdiagnosing which one you hit is expensive in both directions. Treat a configured key limit as an upstream problem and you'll ship retry logic that hammers a ceiling that never moves. Treat insufficient_quota as a transient limit and you'll burn hours retrying an account with no credit in it.

Cause 1: the upstream provider is rate-limiting you

Model vendors cap requests-per-minute and tokens-per-minute. Under load spikes they shed traffic with 429s (and Anthropic-style 529 overload responses) even when your own request rate hasn't changed. This is the one case where client-side retry is the correct fix — but only a bounded retry with three properties:

• Exponential backoff with a cap. Double the wait each attempt, but never beyond a ceiling — unbounded doubling means your fifth retry waits minutes for a limit that resets in seconds.
• Jitter. Randomize each delay. Without it, every client that failed together retries together, and the synchronized wave trips the limit again.
• A give-up path. After a fixed number of attempts, surface the error. An infinite loop is not resilience; it's a queue of stale work.

When the response carries a Retry-After header, honor it — the server is telling you exactly when capacity returns, and guessing with backoff when you've been given the answer only prolongs the outage. Here is the loop in Python, using the standard OpenAI client pointed at Router One:

import os
import random
import time

from openai import OpenAI, RateLimitError

client = OpenAI(
    base_url="https://api.router.one/v1",
    api_key=os.environ["ROUTER_ONE_API_KEY"],
)

MAX_ATTEMPTS = 5
BASE_DELAY = 1.0  # seconds
MAX_DELAY = 30.0  # cap: never wait longer than this


def chat_with_backoff(**kwargs):
    for attempt in range(1, MAX_ATTEMPTS + 1):
        try:
            return client.chat.completions.create(**kwargs)
        except RateLimitError as err:
            if attempt == MAX_ATTEMPTS:
                raise  # give-up path: surface the error, don't loop forever

            retry_after = err.response.headers.get("retry-after")
            if retry_after is not None:
                # the server said exactly when capacity returns — no jitter
                time.sleep(min(float(retry_after), MAX_DELAY))
            else:
                delay = min(BASE_DELAY * 2 ** (attempt - 1), MAX_DELAY)
                # full jitter: spread synchronized clients apart
                time.sleep(random.uniform(0, delay))


response = chat_with_backoff(
    model="gpt-5.5",
    messages=[{"role": "user", "content": "ping"}],
)

Five attempts, a 30-second cap, jittered delays, Retry-After respected, and a clean failure at the end. That's the whole pattern — resist the urge to make it cleverer.

Cause 2: you hit a limit you configured yourself

Router One API keys carry optional per-key controls: rateLimit (requests) and tokenLimitTpm (tokens per minute), alongside maxSpend as a hard spend cap. They exist so a leaked key or a runaway loop can't drain your wallet — the per-customer key pattern leans on them heavily. But a limit you set six weeks ago looks exactly like an upstream 429 to the client that hits it.

The tell: these 429s are deterministic. They start the moment your traffic crosses the configured threshold and recur at the same request rate every time, regardless of time of day or which model you call. No amount of backoff tuning changes the ceiling — the fix is Dashboard → API Keys, where you raise or remove the limit on that key. (If the key stopped with a 402 instead, that's the maxSpend cap or wallet balance — money, not speed. The quick reference covers the distinction.)

Before raising a limit, ask why it's being hit. If one background job is saturating a key that production traffic shares, the better fix is a second key with its own limits, so the noisy workload is isolated and separately visible.

Cause 3: insufficient_quota — billing wearing a 429 costume

If you call the official OpenAI API directly and the error body says insufficient_quota, stop retrying. The status code is 429, but nothing about it is a rate limit: your prepaid billing credit is exhausted, and the "limit" will not reset in seconds, minutes, or ever — not until billing changes. Every retry is a wasted request, and backoff just wastes the same requests more slowly.

This error comes from OpenAI's platform, not from a gateway. The insufficient_quota fix guide walks through the causes and the resolution paths, including the case where the account has a payment method but the credit balance is still zero.

How to tell which one you hit

Guessing from client-side symptoms is unreliable; the request trace isn't. On Router One, Dashboard → Logs shows the final record for every request — status, model, tokens, cost, latency, and the routing decision — filterable by model and time range. The diagnosis reduces to three checks:

1. Read the error body. insufficient_quota names itself. If the string is there, it's billing — go to cause 3.
2. Check the key's configuration. If the failing key has rateLimit or tokenLimitTpm set, compare the configured threshold to your actual request rate in the logs. A match means cause 2, and the fix is the setting.
3. Look at the pattern. Upstream limits (cause 1) are bursty and correlated with load spikes — 429s cluster in specific windows and clear on their own. Configured limits are flat and predictable.

The same per-request records answer the follow-up question — what did the failed window cost you in retried tokens — which is cost tracking territory.

Retry storms: how naive retries make a rate limit worse

A rate limit is a signal that the system is at capacity. The naive response — retry immediately, retry forever — converts that signal into an amplifier: every rejected request becomes two requests, then four, and the aggregate arrival rate climbs precisely when the system needs it to fall. This is a retry storm, and it can hold a service under water long after the original spike has passed. Bounded attempts and jitter (above) are the retry-side defense.

The structural fix is concurrency shaping: bound how many requests are in flight at once, with a semaphore or a fixed worker pool, instead of launching one task per item and letting backoff absorb the damage. A pool of 8 workers draining a queue of 10,000 jobs produces a steady, predictable request rate that stays under the limit by construction. The same 10,000 jobs launched concurrently produce a wall of 429s followed by a synchronized retry wave. Backoff is the seatbelt; a bounded pool is driving at a survivable speed.

Where gateway fallback fits

Routing through Router One adds one more layer before the error reaches you: an eligible 429 or 529 can be retried on another healthy provider serving the same requested model, when such a route exists. The hedges matter — not every 429 is retryable (a configured key limit or an insufficient_quota-style billing failure isn't a candidate), and fallback is not a zero-downtime guarantee: when no compatible route completes the request, your app still receives the error. Keep the client-side backoff loop regardless; how fallback decides is documented separately.

On Router One's own posture: there is no low fixed cap on normal paid usage — abuse prevention, per-account protection limits, and upstream constraints may still apply. If a legitimate workload is hitting a ceiling, that's a conversation with support, not an architecture problem.

FAQ

How do I know if a 429 came from my key's limit or from the upstream provider? Check the request trace in Dashboard → Logs and the key's configuration. If the failing key has rateLimit or tokenLimitTpm set and your logged request rate matches that threshold, it's your configured limit — fix it in the dashboard. Upstream 429s are bursty, correlate with load spikes, and clear on their own; configured limits recur deterministically at the same rate.

Should I retry every 429? No. Retry with bounded exponential backoff and jitter only when the limit can actually reset — an upstream rate limit. A 429 caused by your own key's rateLimit or tokenLimitTpm setting won't clear until the setting changes, and OpenAI's insufficient_quota is exhausted billing credit that no retry schedule will refill.

Does Router One impose its own rate limits on my requests? There is no low fixed cap for normal paid usage. Abuse prevention, per-account protection limits, and upstream provider constraints may still apply. Per-key rateLimit and tokenLimitTpm ceilings are optional controls — they only apply to a key you configured them on.

Is exponential backoff enough on its own? It's necessary but not structural. Backoff handles the individual failed request; it doesn't stop an unbounded batch job from generating the 429s in the first place. Bound your in-flight concurrency with a semaphore or fixed worker pool so the aggregate request rate stays under the limit by construction, and keep backoff as the safety net.

Can gateway fallback make 429s disappear entirely? No. An eligible 429 or 529 can be retried on another healthy provider serving the same requested model when one is available, which absorbs many upstream limit events — but not every 429 is retryable, and it is not a zero-downtime guarantee. Your app should still handle a final 429 with bounded backoff.

The checklist

When a 429 shows up: read the error body first (insufficient_quota means billing), check the key's configured limits second (a dashboard fix), and only then treat it as a genuine upstream limit — bounded backoff with jitter, Retry-After honored, concurrency capped. The error-code hub covers the other status codes, and the API docs cover key configuration end to end. One base URL change gets you the request traces that make the diagnosis a lookup instead of a guess.