Configure LlamaIndex OpenAILike for Router One

Install the OpenAILike integration and create a dedicated key

pip install llama-index-core llama-index-llms-openai-like installs the pieces this guide uses: llama-index-llms-openai-like 0.8.1 (October 1, 2026) needs Python 3.10 or newer and depends on llama-index-llms-openai 0.8.x, whose OpenAI client sends the requests. Create a Router One key for this application alone and give it a maxSpend cap. A query engine or an agent sends requests you never see one by one, and one question can cost several calls, so the cap is the hard stop, and a separate key lets you filter Dashboard → Logs down to this application's traffic.

pip install llama-index-core llama-index-llms-openai-like
export ROUTER_ONE_API_KEY=sk-your-router-one-key

Configure LlamaIndex to use the Router One base URL

Build an OpenAILike with the gateway base URL, the key and an exact chat model ID from /models, and set the three fields its defaults get wrong for a gateway model: is_chat_model=True, context_window from the model page, and is_function_calling_model=True only for IDs whose page lists tool calling. Assign it to Settings.llm so every index, query engine and agent uses it, or pass llm= to a single component. Pass api_base and api_key explicitly, as below: LlamaIndex falls back to the OPENAI_API_BASE and OPENAI_API_KEY variables, not OPENAI_BASE_URL, so a variable left over from another tool can send the calls somewhere else. max_tokens is sent as max_tokens, which Router One's Chat Completions accepts as the output cap:

import os
from llama_index.core import Settings
from llama_index.llms.openai_like import OpenAILike

Settings.llm = OpenAILike(
    model="anthropic/claude-sonnet-5",  # exact ID from /models
    api_base="https://api.router.one/v1",
    api_key=os.environ["ROUTER_ONE_API_KEY"],
    is_chat_model=True,  # the default False sends /v1/completions
    is_function_calling_model=True,  # only if the model page lists tool calling
    context_window=1000000,  # the model page's window, rounded down
    max_tokens=2048,  # sent as max_tokens
    timeout=300.0,  # default 60 seconds
)

print(Settings.llm.complete("Reply with OK"))

Set the three fields OpenAILike cannot learn from the gateway

is_chat_model defaults to False, and then both complete() and chat() go out as the legacy Completions API, POST /v1/completions, with the chat messages flattened into one prompt; Router One does not serve that path, so the call fails with 404. With True, every call is POST /v1/chat/completions. context_window defaults to 3,900 tokens, and LlamaIndex packs retrieved chunks and chat history to fit it, so at the default a query engine truncates, or splits synthesis into extra refine calls, even on a model with a 1M-token window. Take the window from the model's page on /models, which shows it rounded (1.05M, 200K), and write it as a whole number rounded down: 1000000 for anthropic/claude-sonnet-5, openai/gpt-5.6-sol and deepseek-v4-flash, 200000 for anthropic/claude-haiku-4.5. A larger window lets LlamaIndex send longer prompts: on a plan, a request whose input exceeds a model's long-context threshold draws more than one plan request, and wallet billing follows the model's own long-context price line where it has one (see /pricing), so a smaller window can be the cheaper choice for RAG. is_function_calling_model defaults to False; with it, FunctionAgent stops with ValueError: LLM must be a FunctionCallingLLM, and AgentWorkflow.from_tools_or_functions falls back to a text-based ReActAgent. Set it to True only for IDs whose model page lists tool calling.

What the requests carry: temperature, reasoning_effort and tools

Every request carries temperature 0.1, LlamaIndex's default. llama-index-llms-openai forces temperature 1.0, and sends max_completion_tokens instead of max_tokens, only for the unprefixed OpenAI reasoning names on its own list, such as gpt-5.5; a catalog ID with a vendor prefix, such as openai/gpt-5.6-sol, never matches, so it goes out with temperature 0.1 and max_tokens as you set them. If a 400 in Dashboard → Logs names temperature, that is the model rejecting the value: set temperature=1.0 on the OpenAILike instead of changing the base URL. The reasoning_effort argument is forwarded only for those same unprefixed names, so for a catalog ID pass it through additional_kwargs={"reasoning_effort": "low"}. Router One does not turn reasoning_effort into a thinking setting for Claude IDs, except on Claude Opus 5.5, where it maps the value to output_config.effort (none and minimal become low); other models may ignore it or reject the request with 400. With is_function_calling_model=True, tool definitions go out with strict false by default and parallel_tool_calls false unless you allow parallel calls. If the gateway rejects a tool request with HTTP 400 and has no chat candidate that supports requested capabilities: tool_calling, the chosen ID cannot take tool calls; switch models.

Keep embeddings on a provider that serves them

Router One has no embeddings endpoint: POST /v1/embeddings returns 404 even with a valid key. VectorStoreIndex.from_documents and every retriever call Settings.embed_model, which defaults to OpenAIEmbedding (text-embedding-ada-002) and resolves its endpoint the same way the LLM does: the api_base argument first, then OPENAI_API_BASE. Export OPENAI_API_BASE=https://api.router.one/v1 for the whole process and indexing goes to Router One and fails with 404; export only OPENAI_API_KEY with a Router One key and indexing goes to api.openai.com with a key OpenAI rejects (401). Set Settings.embed_model explicitly, either OpenAIEmbedding with an OpenAI key and base URL of its own or a local embedding model, and give the Router One URL and key only to the LLM, as in the example below. Retrieval then runs on the embedding provider and only the synthesis calls reach Router One.

# pip install llama-index-embeddings-openai
import os
from llama_index.core import Settings, SimpleDirectoryReader, VectorStoreIndex
from llama_index.embeddings.openai import OpenAIEmbedding

# Embeddings stay on a provider that serves them (here OpenAI, with its own key)
Settings.embed_model = OpenAIEmbedding(
    model="text-embedding-3-small",
    api_key=os.environ["OPENAI_EMBEDDINGS_KEY"],
    api_base="https://api.openai.com/v1",
)
# Settings.llm is the OpenAILike on Router One from the example above

index = VectorStoreIndex.from_documents(SimpleDirectoryReader("docs").load_data())
print(index.as_query_engine().query("What does the setup guide say about keys?"))

Retries, timeouts and long answers

OpenAILike defaults to max_retries=3 and timeout=60 seconds. It passes max_retries to the OpenAI client it builds, which retries connection errors and HTTP 408, 409, 429 and 5xx, and it also wraps each call in its own retry for connection errors, timeouts, rate limits and server errors, stopping after three attempts or 60 seconds. One failing call can therefore reach the gateway several times, and each client attempt the gateway forwards to a model is its own request with its own request_id in Dashboard → Logs, after Router One has already tried other lines of the same model for a 429, 5xx or timeout. Lower max_retries for batch indexing jobs so one bad input does not multiply. For long answers, stream (stream_chat, astream_chat, stream_complete) or raise timeout. A stream: false call that runs longer than about 25 seconds gets HTTP 200 from the gateway while the answer is prepared; if it then fails, the error arrives as a body without choices, and LlamaIndex reads choices[0] directly, so it surfaces as a TypeError far from the cause. The OpenAI SDK guide shows that response and how to check for it.

GPT-family IDs on the Responses API (optional)

The same package ships OpenAILikeResponses, which sends POST /v1/responses. Use it only for IDs whose model page lists that endpoint: the GPT-family models, deepseek-v4.1-flash and deepseek-v4-flash, and Grok chat models such as grok-4.7. A Claude-family ID sent there is rejected with HTTP 400 model '<id>' must be called via /v1/messages or /v1/chat/completions, and Gemini IDs are not served there either, so keep OpenAILike when one application mixes families. OpenAILikeResponses always counts as a chat model; set context_window and is_function_calling_model as above, and max_output_tokens instead of max_tokens. It sends store false unless you set track_previous_responses=True, which stores each response and chains the next call with previous_response_id; Router One accepts previous_response_id on those IDs.

Which model ID should LlamaIndex send?

Copy the exact model ID from /models, preserving case, hyphens, and version suffixes; do not substitute a display name. Open its detail page and match the supported API endpoints, context window, and capabilities such as tool calling to the provider and features selected in LlamaIndex. A catalog listing does not mean the client can use every feature of that model. Give each client or application a dedicated API key with a maxSpend cap.

Which API protocol is LlamaIndex using?

OpenAI-compatible describes an interface format; it does not make Chat Completions (/v1/chat/completions), Responses (/v1/responses), and Anthropic Messages (/v1/messages) interchangeable. Check the installed client version, provider configuration, and actual request path against the model detail page and API compatibility fact sheet. A successful plain-text chat does not establish support for hosted tools, conversation state, or file-editing features.

Verify the LlamaIndex call in your request trace

Send a simple text request from LlamaIndex, then match its trace in Dashboard → Logs by time, model, and request_id: tokens, cost, latency, and status. Next, test streaming, tool calls, and multi-turn history separately. For failures, retain the actual request path, full error message, and request_id. If there is no matching log, either the request never reached the gateway (client configuration or connectivity) or the gateway refused it before calling a model — a 401 for the key, a 402 for funds, a 429 from your key's or account's limits, or a 400 or 404 for the model ID or endpoint; a streamed request that failed before any output may have no log either. The status and error message the client received tell which, so check them before attributing the error to the gateway or upstream.

FAQ