LLM API 429 Errors: The Three Causes and How to Fix Each

HTTP 429 is the least informative error an LLM API returns, because three genuinely different failures share the same status code — and each one has a different fix. Retry code fixes exactly one of them. For the second, retrying just delays the inevitable, and for the third, retrying is pure waste. This post is the deep dive behind the error-code quick reference: how to tell the three apart, the production-grade retry loop for the one case where retrying helps, and the structural change that stops 429s from happening in the first place.

One status code, three different failures

Misdiagnosing which one you hit is expensive in both directions. Treat a gateway limit as a passing upstream problem and you'll ship retry logic that hammers a ceiling that never moves. Treat insufficient_quota as a transient limit and you'll burn hours retrying an account with no credit in it.

Cause 1: the upstream provider is rate-limiting you

Model vendors cap requests-per-minute and tokens-per-minute. Under load spikes they shed traffic with 429s (and Anthropic-style 529 overload responses) even when your own request rate hasn't changed. This is the one case where client-side retry is the correct fix — but only a bounded retry with three properties:

• Exponential backoff with a cap. Double the wait each attempt, but never beyond a ceiling — unbounded doubling means your fifth retry waits minutes for a limit that resets in seconds.
• Jitter. Randomize each delay. Without it, every client that failed together retries together, and the synchronized wave trips the limit again.
• A give-up path. After a fixed number of attempts, surface the error. An infinite loop is not resilience; it's a queue of stale work.

When the response carries a Retry-After header, honor it — the server is telling you exactly when capacity returns, and guessing with backoff when you've been given the answer only prolongs the outage. Here is the loop in Python, using the standard OpenAI client pointed at Router One:

import os
import random
import time

from openai import OpenAI, RateLimitError

client = OpenAI(
    base_url="https://api.router.one/v1",
    api_key=os.environ["ROUTER_ONE_API_KEY"],
)

MAX_ATTEMPTS = 5
BASE_DELAY = 1.0  # seconds
MAX_DELAY = 30.0  # cap: never wait longer than this


def chat_with_backoff(**kwargs):
    for attempt in range(1, MAX_ATTEMPTS + 1):
        try:
            return client.chat.completions.create(**kwargs)
        except RateLimitError as err:
            if attempt == MAX_ATTEMPTS:
                raise  # give-up path: surface the error, don't loop forever

            retry_after = err.response.headers.get("retry-after")
            if retry_after is not None:
                # the server said exactly when capacity returns — no jitter
                time.sleep(min(float(retry_after), MAX_DELAY))
            else:
                delay = min(BASE_DELAY * 2 ** (attempt - 1), MAX_DELAY)
                # full jitter: spread synchronized clients apart
                time.sleep(random.uniform(0, delay))


response = chat_with_backoff(
    model="openai/gpt-5.5",
    messages=[{"role": "user", "content": "ping"}],
)

Five attempts, a 30-second cap, jittered delays, Retry-After respected, and a clean failure at the end. That's the whole pattern — resist the urge to make it cleverer.

Cause 2: you hit a gateway limit on the key or the account

Every Router One API key has a request-per-minute limit (rateLimit) and a token-per-minute limit (tokenLimitTpm), and all keys of one account also share account-level request and token limits, including a per-model request limit. They start at platform defaults — there is no low fixed cap for normal paid usage — and exist so a leaked key or a runaway loop can't flood the account. Unlike maxSpend, the hard spend cap you set per key in Dashboard → API Keys, these limits are not edited in the dashboard. The per-customer key pattern leans on the per-key spend cap; the throughput limits sit underneath it.

The tell: the response names the limit. A gateway 429 carries X-RateLimit-Scope — api_key for the key's limit, subject for the account-wide limit, subject_model for the account's use of one model — along with X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset and Retry-After; the error code is RATE_LIMIT_EXCEEDED for requests or TOKEN_QUOTA_EXCEEDED for tokens per minute. These 429s are deterministic: they recur at the same rate regardless of time of day. The token limit counts an estimate when a request starts (prompt size plus max_tokens) and corrects it when the request settles, so a burst of large prompts can reach it before actual usage does. (If the key stopped with a 402 instead, that's the maxSpend cap or wallet balance — money, not speed. The quick reference covers the distinction.)

Before asking for a higher limit, ask why it's being hit. If one background job saturates a key that production traffic shares, the better fix is a second key, so the noisy workload is isolated and separately visible — though keys of one account still share the account-level limit. When the workload genuinely needs more — a benchmark, a batch backfill, a relay's aggregate traffic — key and account limits can be raised on request: email support@router.one with your account ID, the key, the models, and the peak requests and tokens per minute you expect. Upstream capacity still applies above the gateway limit.

Cause 3: insufficient_quota — billing wearing a 429 costume

If you call the official OpenAI API directly and the error body says insufficient_quota, stop retrying. The status code is 429, but nothing about it is a rate limit: your prepaid billing credit is exhausted, and the "limit" will not reset in seconds, minutes, or ever — not until billing changes. Every retry is a wasted request, and backoff just wastes the same requests more slowly.

This error comes from OpenAI's platform, not from a gateway. The insufficient_quota fix guide walks through the causes and the resolution paths, including the case where the account has a payment method but the credit balance is still zero.

How to tell which one you hit

Guessing from client-side symptoms is unreliable; the request trace isn't. On Router One, Dashboard → Logs shows the final record for every request — status, model, tokens, cost, latency, and request ID — filterable by model and time range. The diagnosis reduces to three checks:

1. Read the error body. insufficient_quota names itself. If the string is there, it's billing — go to cause 3.
2. Read X-RateLimit-Scope. api_key, subject or subject_model means cause 2 — a gateway limit on the key or the account, with X-RateLimit-Limit telling you where the ceiling sits. upstream_provider points at cause 1.
3. Look at the pattern. Upstream limits (cause 1) are bursty and correlated with load spikes — 429s cluster in specific windows and clear on their own. Gateway limits are flat and predictable.

The same per-request records answer the follow-up question — what did the failed window cost you in retried tokens — which is cost tracking territory.

Retry storms: how naive retries make a rate limit worse

A rate limit is a signal that the system is at capacity. The naive response — retry immediately, retry forever — converts that signal into an amplifier: every rejected request becomes two requests, then four, and the aggregate arrival rate climbs precisely when the system needs it to fall. This is a retry storm, and it can hold a service under water long after the original spike has passed. Bounded attempts and jitter (above) are the retry-side defense.

The structural fix is concurrency shaping: bound how many requests are in flight at once, with a semaphore or a fixed worker pool, instead of launching one task per item and letting backoff absorb the damage. A pool of 8 workers draining a queue of 10,000 jobs produces a steady, predictable request rate that stays under the limit by construction. The same 10,000 jobs launched concurrently produce a wall of 429s followed by a synchronized retry wave. Backoff is the seatbelt; a bounded pool is driving at a survivable speed.

Where gateway fallback fits

Routing through Router One adds one more layer before the error reaches you: an eligible 429 or 529 can be retried on another healthy provider serving the same requested model, when such a route exists. The hedges matter — not every 429 is retryable (a gateway limit on the key or account, or an insufficient_quota-style billing failure, isn't a candidate), and fallback is not a zero-downtime guarantee: when no compatible route completes the request, your app still receives the error. Keep the client-side backoff loop regardless; how fallback decides is documented separately.

On Router One's own posture: there is no low fixed cap on normal paid usage — abuse prevention, per-account protection limits, and upstream constraints may still apply. If a legitimate workload is hitting a ceiling, that's a conversation with support, not an architecture problem.

FAQ

How do I know if a 429 came from my key's limit or from the upstream provider? Read the X-RateLimit-Scope response header. api_key, subject or subject_model means a gateway limit on the key, the account, or the account's use of one model, with the error code RATE_LIMIT_EXCEEDED or TOKEN_QUOTA_EXCEEDED; upstream_provider means the model vendor's limit. Upstream 429s are bursty, correlate with load spikes, and clear on their own; gateway limits recur deterministically at the same rate. Dashboard → Logs shows the status and time of every request.

Should I retry every 429? No. Retry with bounded exponential backoff and jitter only when the limit can actually reset — an upstream rate limit. A gateway limit resets every minute, so Retry-After pacing works, but sustained traffic above it keeps failing until the load drops or the limit is raised; OpenAI's insufficient_quota is exhausted billing credit that no retry schedule will refill.

Does Router One impose its own rate limits on my requests? There is no low fixed cap for normal paid usage, but every key and every account runs under default request-per-minute and token-per-minute limits, and upstream provider constraints still apply. The limits are not edited in the dashboard; they can be raised on request by email to support@router.one with your account ID, the key, and your expected peak traffic.

Is exponential backoff enough on its own? It's necessary but not structural. Backoff handles the individual failed request; it doesn't stop an unbounded batch job from generating the 429s in the first place. Bound your in-flight concurrency with a semaphore or fixed worker pool so the aggregate request rate stays under the limit by construction, and keep backoff as the safety net.

Can gateway fallback make 429s disappear entirely? No. An eligible 429 or 529 can be retried on another healthy provider serving the same requested model when one is available, which absorbs many upstream limit events — but not every 429 is retryable, and it is not a zero-downtime guarantee. Your app should still handle a final 429 with bounded backoff.

The checklist

When a 429 shows up: read the error body first (insufficient_quota means billing), read X-RateLimit-Scope second (a gateway limit on the key or account — spread the load or ask support to raise it), and only then treat it as a genuine upstream limit — bounded backoff with jitter, Retry-After honored, concurrency capped. The error-code hub covers the other status codes, and the API docs cover keys and headers end to end. One base URL change gets you the request traces that make the diagnosis a lookup instead of a guess.