Smart routing methodology
Router One routes an exact-model request to that model by default (the model_name strategy): the routes serving that model are tried in a gateway-managed order, a 429, 5xx or timeout on one route can be retried on another route serving the same model, and routes with a sustained upstream-error rate are moved to the back of the order until they recover. Only model="auto" lets the gateway choose the model: it classifies each request into a low, medium or high tier and serves it from that tier's server-owned model list. Routing weights are not customer-configurable.
Last updated:
Routing modes and route order
- Exact model (default)
- Production runs the model_name strategy. A request that names a model is served by that model, and only routes serving that same model are candidates; failover never substitutes another model or variant.
- Route order
- The routes serving a model are tried in a gateway-managed priority order. The order is not customer-configurable, and a route that is cooling down after a failure is skipped while other routes are available.
- Failing-route demotion
- Since 2026-09-24, a route whose recent upstream errors (bad-gateway, unavailable and timeout responses) stay high is moved to the back of its model's order and kept only as a last-resort fallback. It returns to its normal place once its error rate recovers.
- model="auto"
- The gateway classifies the request with rules plus a lightweight classifier into a low, medium or high tier and serves it with a model from that tier's server-owned list. An optional X-Route-Session header keeps a conversation on the same choice for a while. Tier lists are managed server-side and can change.
Fallback behavior
- Trigger
- A 429, a 5xx or a timeout from one route can be retried on another route serving the same model when one is available. Not every error is retryable, and a request still fails when no route completes it.
- Same requested model
- For an exact-model request, fallback can try another healthy provider route serving that same model and does not substitute another model variant. The model="auto" path instead retries within the candidates of its chosen tier.
- Cooldowns
- A route that returns a 429 cools down (for the upstream retry-after when one is sent), a route that returns a bad-gateway, unavailable or timeout error cools down briefly, and a route that fails repeatedly is taken out for a cooldown, then probed before it returns.
- Fallback latency
- End-to-end latency includes the failed attempt and any subsequent attempt on another route. It varies with the upstream failure mode and response time; Router One does not publish a universal 200ms fallback guarantee.
- Bounded retries
- Retries are limited by the routes available for the model and by gateway policy; model="auto" retries under a bounded retry budget. The public product contract does not expose a per-project retry cap.
What appears in your trace
- Model and request ID
- The recorded model and request ID identify the call for usage review and support.
- Status and recorded latency
- Customer Logs shows request status, recorded latency, and any sanitized error. Upstream provider identities, service routes, and intermediate attempt chains are not disclosed.
- Token counts and cost
- Input and output tokens, available cache-read tokens, and settled cost or a pending label. Pricing details appear when supplied.
Configuration boundary
- Exact model selection
- Specify the model you want in the standard OpenAI-compatible request. The default model_name strategy keeps that choice fixed on every request.
- Auto mode
- With model="auto", Router One owns the classification and the tier lists, and retries eligible failures within the chosen tier under a bounded retry budget.
- No implied custom controls
- This methodology does not promise per-project or per-key weights, quality scores, a request-body router object, provider allowlists, or a project-level fallback switch.
FAQ
Can I configure routing weights per project or API key?
No. The current public product contract does not expose per-project or per-key routing weights. Route order is managed by the gateway: the routes serving the requested model are tried in a managed priority order, with failing routes moved to the back.
Is Router One swapping models silently?
No. Provider fallback stays on the requested model. The trace records the model and request ID; the public contract does not promise substitution to another model variant.
How quickly does routing react to a failing route?
Within the request, a 429, 5xx or timeout can move it to another route serving the same model. For later requests, a route that just failed cools down, and a route with a sustained upstream-error rate stays at the back of the order until its error rate recovers. No fixed number of seconds or requests is guaranteed.