One LlamaIndex LLM class for every model family

LlamaIndex is the data framework for RAG and agents over your own documents, and a single query can fan out into many LLM calls — answer synthesis, sub-questions, reranking, agent steps. Its OpenAILike class talks to any OpenAI-compatible endpoint, so one configuration pointed at Router One reaches GPT, Claude, Gemini, and DeepSeek family models by changing the model string — and every one of those calls gets a per-request cost trace on the gateway, without adding callbacks.

Get your API key

Configure LlamaIndex to use the Router One base URL

Install llama-index-llms-openai-like and build an OpenAILike with the gateway base URL, your key, and the exact model ID from /models. Set is_chat_model to True (every text model in the catalog is a chat model), is_function_calling_model to True for models that support tools, and context_window to the window shown on /models — left unset it falls back to a 3900-token default. Assign the instance to Settings.llm, or pass it to any index, query engine, or agent:

llamaindex-router-one.py

# pip install llama-index llama-index-llms-openai-like
from llama_index.core import Settings
from llama_index.llms.openai_like import OpenAILike

Settings.llm = OpenAILike(
    model="<model-id-from-/models>",
    api_base="https://api.router.one/v1",
    api_key="sk-your-router-one-key",
    is_chat_model=True,
    is_function_calling_model=True,
    context_window=200000,  # copy the window shown on /models
)

print(Settings.llm.complete("Hello!"))

Which model ID should LlamaIndex send?

Copy the exact model ID from the /models page — IDs are case-sensitive, and the page lists each model's context window, capabilities, and current per-token rates. Create a dedicated API key per tool with its own maxSpend cap, so one runaway tool can't affect other workloads.

Verify the LlamaIndex call in your request trace

After your first request, open Dashboard → Logs to see its full trace: model, tokens, cost, latency, and status code. From here every LlamaIndex call has a ledger and a trail instead of being a black box.

FAQ

Why OpenAILike instead of the OpenAI class with api_base?

Because the plain OpenAI class checks the model name against OpenAI's own model list to infer its context size, and raises ValueError: Unknown model for anything else — including every non-GPT catalog ID — as soon as it needs that metadata. OpenAILike is the documented route for OpenAI-compatible endpoints: it skips the lookup and takes the model's metadata from the arguments you pass, which is why is_chat_model and context_window are set explicitly.

Is there a TypeScript equivalent?

Yes. In LlamaIndex.TS the OpenAI class from @llamaindex/openai takes baseURL and apiKey in its constructor — new OpenAI({ model, apiKey, baseURL }) with the gateway URL — or reads OPENAI_BASE_URL and OPENAI_API_KEY from the environment. Assign it to Settings.llm and the same model-string switching applies.

Which models can LlamaIndex use through the gateway?

Any chat-capable model in the catalog — GPT, Claude, Gemini, Grok, DeepSeek, GLM, MiniMax, and Doubao families. The /models page is the source of truth for model IDs and per-token rates.

Does this work from Mainland China?

Yes. The gateway is reachable from Mainland China without a VPN, and the configuration is identical to the global setup.

How do I debug a 401/403/429?

Open Dashboard → Logs first to see whether requests reached the gateway and with what status, then walk the error-codes reference to check env vars, key status, and limits.

Get your API key