Connect DSPy to Router One through dspy.LM, the openai/ prefix, and one api_base

Install dspy and set your credentials

Use Python 3.10 or newer: the package metadata on PyPI requires >=3.10 and <3.15, and the DSPy setup page says it works in Python 3.10+ environments. pip install dspy (or uv add dspy) is the documented install; litellm and the openai package are declared dependencies, so nothing else is needed for an OpenAI-compatible endpoint. pip picks the latest stable release, 3.3.1 when this guide was checked; 3.4.0b1 is a pre-release that pip skips unless you ask for it. The shell example below is for macOS/Linux. Replace both placeholders before running it, using a Router One key and the exact ID of a current chat model. The ROUTER_ONE_* names belong to this example, which reads them explicitly; OPENAI_API_KEY is not needed, because the key is passed to dspy.LM as api_key, and the api_key and api_base arguments are used even when OPENAI_API_KEY or OPENAI_API_BASE is set in the environment. Keep the same environment active when running the Python file.

python -m pip install -U dspy
export ROUTER_ONE_API_KEY="sk-your-router-one-key"
export ROUTER_ONE_MODEL_ID="<exact-model-id-from-/models>"

Configure DSPy to use the Router One base URL

Save this as program.py and run python program.py. dspy.LM takes a LiteLLM-style provider/model string, and its first segment must be openai/: the LiteLLM docs define that prefix as the instruction to call an OpenAI /chat/completions endpoint, and the string is split at the first slash only, so everything after openai/ is sent unchanged as the request's model field. The example builds it from ROUTER_ONE_MODEL_ID, which is why a GPT-family ID such as openai/gpt-5.5 becomes openai/openai/gpt-5.5 and a Claude ID becomes openai/anthropic/claude-sonnet-5. api_base is the /v1 base URL with nothing appended; the OpenAI client underneath adds /chat/completions itself. api_key is the Router One key. model_type='chat' is the default and is written out only to make the endpoint explicit. cache=False is there for the first runs: DSPy caches responses in memory and on disk by default, so without it a second run of the same file is answered locally and sends nothing. dspy.configure(lm=lm, track_usage=True) makes this LM the default for every module and turns on token accounting, which result.get_lm_usage() returns per model string. dspy.Predict sends one request for the signature question -> answer; dspy.ChainOfThought adds a reasoning output field and is still one request; dspy.ReAct sends one request per iteration and one more that extracts the final answer, while the add function runs in your process. In 3.3.1 temperature and max_tokens default to None and are left out of the request, so the model's own defaults apply. dspy.inspect_history(n=1) prints the last prompt and reply as DSPy sent them, and lm.history keeps one record per call with its usage:

import os

import dspy

# openai/ selects the OpenAI Chat Completions route; the rest of the string is sent
# unchanged as the model field (openai/openai/gpt-5.5 for a GPT-family ID).
lm = dspy.LM(
    "openai/" + os.environ["ROUTER_ONE_MODEL_ID"],
    api_base="https://api.router.one/v1",  # ends in /v1, nothing after it
    api_key=os.environ["ROUTER_ONE_API_KEY"],
    model_type="chat",  # POST /v1/chat/completions
    cache=False,  # first runs: every call reaches the gateway and shows up in Logs
)
dspy.configure(lm=lm, track_usage=True)

# 1. One Predict call is one request
qa = dspy.Predict("question -> answer")
result = qa(question="What is the capital of France?")
print(result.answer, result.get_lm_usage())

# 2. ChainOfThought adds a reasoning output field; still one request
cot = dspy.ChainOfThought("question -> answer")
print(cot(question="What is 17 * 23?").answer)


# 3. ReAct: one request per iteration plus one that extracts the answer; the tool runs in your process
def add(a: int, b: int) -> int:
    """Add two integers."""
    return a + b


agent = dspy.ReAct("question -> answer", tools=[add], max_iters=5)
print(agent(question="What is 1234 + 4321?").answer)

# The last call as DSPy sent it, and the per-call records DSPy keeps
dspy.inspect_history(n=1)
print(len(lm.history), lm.history[-1]["usage"])

Which DSPy setting sends which request

dspy.LM carries the connection; modules and adapters decide what each request looks like. The rows give the value for each setting in this guide, the request it produces, and what to confirm before relying on it.

Budget an optimizer run and reconcile it with Logs

A DSPy program at inference time is a handful of requests; compile() is where spend concentrates. An optimizer runs your program over the training and validation examples many times: the DSPy FAQ reports one BootstrapFewShotWithRandomSearch compile at about 3,200 API calls, 2.7 million input tokens, and 156,000 output tokens, and the optimizer guide calls compile() expensive for MIPROv2 and GEPA. The budget knobs are client-side and count work, not money: MIPROv2's auto setting (light by default, then medium and heavy) sets 6, 12, or 18 candidates and caps the validation set at 100, 300, or 1,000 examples; GEPA requires exactly one of auto, max_full_evals, or max_metric_calls and logs the approximate number of metric calls before it starts. GEPA's reflection_lm and MIPROv2's prompt_model are separate dspy.LM objects, so give them the same openai/ prefix and api_base, or a key of their own if you want proposal spend separated from task spend. Concurrency comes from num_threads: dspy.Evaluate and the optimizers fall back to dspy.settings.num_threads, which is 8, so up to eight requests are in flight during an evaluation. If the key's rateLimit is lower than that burst, the gateway answers 429 and DSPy raises dspy.LMRateLimitError after its retries; lower num_threads or raise the key's rateLimit rather than adding retries. Retries multiply requests: num_retries defaults to 3, and against a local mock server DSPy 3.3.1 with LiteLLM 1.101 sent a request answered with 429 or 500 seven times, and one answered with 400, 401, 402, or 404 four times, before raising, while num_retries=0 sent it once. Every attempt that reaches the gateway is its own request in Dashboard → Logs with its own request_id. Give each program, and each compile run you want to bound, a dedicated key with maxSpend: at the cap the next call gets HTTP 402, DSPy raises dspy.LMBillingError, the parallel executor stops after max_errors failed examples (10 by default) with the message Execution cancelled due to errors or interruption, and the wallet and your other keys are untouched. Expect fewer rows in Logs than calls in your code while cache=True: identical requests, including a whole repeated compile, are replayed from disk, and the cache key ignores api_key and api_base, so an answer cached for the same model string on another endpoint is replayed too. get_lm_usage() is DSPy's tally of the usage the API reported and is empty for cached calls; the cost field in lm.history is LiteLLM's estimate from its own price list and can be None for IDs it does not know. The charge is what Logs records: reconcile by key, time, exact model, and token counts, and keep the request_id when reporting a failure. Router One serves the model requests and records their traces; the optimizer, the metric, the tools, and the cache all run in your process.

Embeddings and fine-tuning stay outside the gateway

dspy.Embedder accepts either a hosted model string or a Python callable. A hosted string such as openai/text-embedding-3-small goes through LiteLLM's embedding call, and with Router One's api_base that is POST /v1/embeddings, an endpoint the gateway does not serve, so the call fails with a 404 that LiteLLM raises as NotFoundError. Keep embeddings on another path: the Embedder reference shows a local sentence-transformers model passed as a callable, dspy.Embedder(model.encode), and a hosted embedding model from another provider works with that provider's own key and api_base. dspy.retrievers.Embeddings takes the embedder as an argument, so a RAG program can retrieve with a local or third-party embedder and send only its generation calls to Router One through dspy.LM. The two objects are independent: dspy.configure(lm=...) does not touch the embedder, and the embedder's requests never appear in Logs. The same boundary applies to training: optimizers that train weights, such as BootstrapFinetune, need a fine-tuning API, which Router One does not serve, while the prompt optimizers (BootstrapFewShot, MIPROv2, GEPA) need only chat calls.

Which model ID should DSPy send?

Copy the exact model ID from /models, preserving case, hyphens, and version suffixes; do not substitute a display name. Open its detail page and match the supported API endpoints, context window, and capabilities such as tool calling to the provider and features selected in DSPy. A catalog listing does not mean the client can use every feature of that model. Give each tool a dedicated API key with a maxSpend cap.

Which API protocol is DSPy using?

OpenAI-compatible describes an interface format; it does not make Chat Completions (/v1/chat/completions), Responses (/v1/responses), and Anthropic Messages (/v1/messages) interchangeable. Check the installed client version, provider configuration, and actual request path against the model detail page and API compatibility fact sheet. A successful plain-text chat does not establish support for hosted tools, conversation state, or file-editing features.

Verify the DSPy call in your request trace

Send a simple text request from DSPy, then match its trace in Dashboard → Logs by time, model, and request_id: tokens, cost, latency, and status. Next, test streaming, tool calls, and multi-turn history separately. For failures, retain the actual request path, full error message, and request_id. If there is no matching log, check client configuration and connectivity before attributing the error to the gateway or upstream.

FAQ