Short answer: model routing has two layers. Choosing a model trades latency, cost and quality against each other, and that choice is yours — or the gateway's tier classifier's when you send model="auto". Getting the request to that model is about reliability: which route serves it and what happens when a route fails. In Router One, the model you name is the model that serves the request, and failover stays on routes for that same model.
When your application sends a request to an LLM, someone has to decide which model handles it. In a simple setup, that decision is hardcoded: every request goes to the same model at the same provider. In a production system handling thousands of requests per minute across different use cases, that approach leaves performance and money on the table.
What Is Model Routing?
Model routing is the decision layer that sits between your application and the LLM providers. Depending on the routing mode, it either honors an exact model selection or picks a model for you from a server-owned list.
The simplest form is a static mapping: "always use Claude Sonnet." At the model level that is also Router One's default: production traffic uses the model_name strategy, so the model you name is the model that serves the request. What the gateway still decides is which route to that model is tried first, and what happens if it fails. With model="auto", the gateway classifies the request into a low, medium or high tier and serves it with a model from that tier's server-owned list.
The value of routing increases with the number of models and providers you have access to. With one model, there is nothing to route. With ten models across four providers, the routing decision can meaningfully impact cost, speed, and reliability on every single request.
Latency, Cost and Quality: Where Each Trade-off Is Made
The three dimensions in the title are real trade-offs, but they are settled when a model is chosen, not re-computed per request inside the gateway. Router One does not re-rank routes by measured latency or posted price, and it does not grade response quality.
Latency
How fast will this model respond? Latency varies not just between models but between providers of the same model, and it fluctuates throughout the day based on load. A model that responds in 200 milliseconds at 2 AM might take 2 seconds at peak hours.
For latency-sensitive workloads such as chatbots, autocomplete and interactive agents, the lever is model choice: put them on a smaller, faster model and measure on your own traffic. Every request's latency is recorded in the request log, so you can compare models over a time range before you switch.
Cost
How much will this request cost? Pricing varies dramatically across models and providers. The same request can cost an order of magnitude more on a frontier model than on a small one — posted rates are on /models. For high-volume workloads, this difference translates directly to thousands of dollars per month.
Posted per-model rates are the numbers to weigh here, and the choice stays with you: Router One does not move an exact-model request to a cheaper model. Put simple tasks on small models in your own code, or send model="auto" and let the tier classifier choose.
Reliability
Did recent attempts on this route succeed? This is the dimension the gateway manages for you, and the next section explains how.
Response quality remains an application-level model-selection concern. Router One's public routing contract does not expose a quality score or a customer-configurable quality weight.
How Router One Orders the Routes for One Model
A widely used model is usually reachable over more than one upstream route. For an exact-model request, the gateway collects the routes serving that model and tries them in a gateway-set priority order; the first route that answers serves the request. Three mechanisms keep that order honest:
- Retry on another route. If a route returns 429 or a 5xx, or times out, the request is retried on another route for the same model. It is never swapped to a different model.
- Cooldowns. A route that answers 429 cools down before it is tried again, and a route that fails repeatedly is taken out for a cooldown and probed again afterwards.
- Demotion of failing routes. Since 2026-09-24, routes with a sustained upstream-error rate are automatically moved to the back of the order and kept only as a last-resort fallback until their error rate recovers.
Router One does not publish thresholds or window sizes for these mechanisms, and none of them is customer-configurable. They change which route is tried first, never which model answers.
Default and Auto Routing Modes
Router One deliberately separates exact model selection from automatic model selection:
- Default
model_namestrategy — the model id in the standard OpenAI-compatible request is the model that serves it; failover stays on routes for that model. model="auto"— the gateway classifies each request with rules plus a lightweight classifier into a low, medium or high tier, serves it with a model from that tier's server-owned list, and retries eligible failures within those candidates. An optionalX-Route-Sessionheader keeps a conversation on the same choice for a while. The tier lists belong to the server and change as the catalog does.
The public contract does not expose routing weights per project or API key, a quality score, or a custom router object in the request body.
Automatic Failover: Detecting Failures and Rerouting
Provider outages and degradations are not edge cases — they are regular events. Any production system that depends on a single provider without failover is accepting unnecessary downtime risk.
Router One's failover has three boundaries:
Same model only. An eligible failure (429, 5xx or timeout) is retried on another route serving the model you asked for. If no route for that model is available, the request returns an error rather than a different model's answer.
No customer-tunable thresholds. Router One does not publish a fixed rule such as "three 500 errors in ten seconds" as a customer-configurable contract.
Recovery. A route in cooldown is probed again when the cooldown ends, and a demoted route returns to its normal place in the order once its error rate recovers. The public contract does not promise a particular probe schedule.
How Router One Implements Routing
Router One's routing engine keeps its default behavior explicit: production uses the model_name strategy. An exact-model request therefore keeps the model you chose, and routing only decides which route to that model serves it.
With model="auto", the tier classifier chooses the model and eligible failures are retried within the server-owned candidates. Neither mode implies a quality score or project/API-key weight controls.
The customer request log records the model, request ID, token usage, cost, latency, and status. It does not disclose upstream suppliers, channels, or internal routing attempts. Use the request ID when asking support to investigate an issue.
The smart routing page documents the routing modes, automatic LLM fallback covers what happens when a call fails, and routing methodology states what is and is not promised.
FAQ
What is AI model routing?
Model routing is the decision layer between your application and the LLM providers. In Router One it has two modes: by default the exact model you name serves the request and failover stays on routes for that model; with model="auto", the gateway classifies each request into a low, medium or high tier and serves it from that tier's server-owned list.
Does Router One reroute my request when I ask for an exact model?
Not to another model. Production traffic defaults to the model_name strategy, so an exact-model request keeps your model selection; if a route fails with a 429, a 5xx or a timeout, the request is retried on another route for the same model. Tier-based model selection applies only when you send model="auto".
Does Router One rank routes by latency or cost? No. Routes for a model are tried in a gateway-set priority order, and Router One does not re-rank them by measured latency or posted price. What changes the order is failure: routes in cooldown are skipped, and routes with a sustained upstream-error rate move to the back until they recover. There is no quality score and no customer-configurable routing weight.
What happens when a route keeps failing? A route that fails repeatedly is taken out for a cooldown and probed again afterwards. Since 2026-09-24, routes with a sustained upstream-error rate are also moved to the back of the order and kept only as a last-resort fallback until the rate recovers. Router One does not publish the thresholds.
What happens when an upstream call fails? When an upstream failure is eligible for retry (429, 5xx or timeout) and another route for the same model is available, the gateway retries it automatically. Automatic LLM fallback covers what happens when a call fails.
Start Routing Smarter
If you are sending all LLM traffic to a single provider, same-model failover across routes reduces that dependency while keeping exact-model requests predictable.
Sign up at router.one, keep the standard OpenAI-compatible request shape, and use the documented routing modes and request logs to evaluate behavior with your own workload.