# Connect AnythingLLM's Generic OpenAI provider to Router One > Markdown mirror of https://router.one/integrations/anythingllm for AI assistants and crawlers. Router One is an OpenAI-compatible LLM API gateway. > Last updated: 2026-09-11 AnythingLLM is an open-source chat and document-RAG application, shipped as a desktop app and as a Docker image, that answers over your uploads and runs @agent tool sessions. Its Generic OpenAI provider sends Chat Completions requests, so one Router One key gives every workspace any chat model in the catalog, with a cost trace per request in Dashboard → Logs. The embedder and vector database behind document chat stay inside AnythingLLM, because Router One serves chat only. This guide covers the Settings → LLM form and the Docker .env keys, the context-window and Max Tokens fields, and what changes for documents and agents. ## Start AnythingLLM and create a dedicated key Use AnythingLLM Desktop (macOS, Windows, Linux) or the Docker image. The official Docker command below persists data on the host and mounts an .env file at /app/server/.env; the app then opens at http://localhost:3001. Desktop users skip the .env entirely and use the settings form. In Router One, create a key for this instance and give it a maxSpend cap: AnythingLLM sends every workspace's requests with the one key configured for the provider. `terminal` ```bash export STORAGE_LOCATION=$HOME/anythingllm && \ mkdir -p $STORAGE_LOCATION && \ touch "$STORAGE_LOCATION/.env" && \ docker run -d -p 3001:3001 \ --cap-add SYS_ADMIN \ --name anythingllm \ -v ${STORAGE_LOCATION}:/app/server/storage \ -v ${STORAGE_LOCATION}/.env:/app/server/.env \ -e STORAGE_DIR="/app/server/storage" \ mintplexlabs/anythingllm ``` ## Configure AnythingLLM to use the Router One base URL This .env is the file the Docker command mounts at /app/server/.env (a docker-compose build reads docker/.env instead); edit it, then restart the container, because the variables are read at startup. LLM_PROVIDER='generic-openai' selects the Generic OpenAI provider. GENERIC_OPEN_AI_BASE_PATH takes the /v1 base URL: the provider passes it unchanged to the official openai Node SDK, which appends /chat/completions, so every chat request is POST /v1/chat/completions. GENERIC_OPEN_AI_API_KEY is your Router One key, and GENERIC_OPEN_AI_MODEL_PREF the exact catalog ID, including any prefix that is part of the ID. GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT is that model's context window, copied from its detail page; AnythingLLM uses it to budget history, system prompt, and user turn, and the source falls back to 4096 when it is unset. GENERIC_OPEN_AI_MAX_TOKENS is the max_tokens sent with each request (source default 1024); raise it if replies stop mid-sentence. On Desktop, or on Docker without editing files, the Settings → LLM page writes the same keys through its Base URL, API Key, Selected Model, Model context window, and Max Tokens fields: `.env` ```bash # Docker: the file mounted at /app/server/.env (docker/.env for a docker-compose build) # Desktop: enter the same values in Settings → LLM instead of editing a file LLM_PROVIDER='generic-openai' GENERIC_OPEN_AI_BASE_PATH='https://api.router.one/v1' GENERIC_OPEN_AI_API_KEY=sk-your-router-one-key GENERIC_OPEN_AI_MODEL_PREF='' # Context window of that model, copied from its detail page on /models GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT= # max_tokens sent with every request; the source default is 1024 GENERIC_OPEN_AI_MAX_TOKENS=1024 # Keep the built-in embedder: Router One serves no embeddings endpoint EMBEDDING_ENGINE='native' EMBEDDING_MODEL_PREF='Xenova/all-MiniLM-L6-v2' # Restart the container after editing; the variables are read at startup ``` ## Settings form fields and their .env keys Both paths write the same configuration: the settings form is the only path on Desktop, and either works on Docker. The Selected Model field queries the gateway's /models endpoint as soon as Base URL and API Key are filled in and offers the catalog IDs in a dropdown; if that request fails, it turns into a text box that takes the exact ID. AnythingLLM does not read a model's context window from the gateway, so that field is always yours to fill; API Key takes the dedicated key from the setup step: | Settings → LLM | .env key | What to enter or verify | | --- | --- | --- | | Base URL | GENERIC_OPEN_AI_BASE_PATH | https://api.router.one/v1, with /v1; the SDK appends /chat/completions | | Selected Model | GENERIC_OPEN_AI_MODEL_PREF | The exact ID from /models; its detail page must list POST /v1/chat/completions | | Model context window | GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT | The context window shown on the model detail page; 4096 if left unset | | Max Tokens | GENERIC_OPEN_AI_MAX_TOKENS | max_tokens per request, source default 1024; raise it for long answers | | Settings → Embedder | EMBEDDING_ENGINE | Keep native (the built-in embedder) or any embedding provider; not Router One | ## Documents, agents, and what actually reaches the gateway Document chat is RAG inside AnythingLLM: the embedder turns uploads into vectors, the vector database (LanceDB is built in) stores them, and only the assembled prompt with retrieved context reaches Router One as a Chat Completions request. The embedder is set system-wide, and the official docs advise against changing it once documents are embedded, because a switch means re-embedding everything; the built-in all-MiniLM-L6-v2 model runs on CPU without any external API. @agent sessions run AnythingLLM's own skills, such as web browsing and web scraping, inside the app; the provider treats tool calling as available unless PROVIDER_DISABLE_NATIVE_TOOL_CALLING lists generic-openai, so choose a model whose detail page confirms tool calling on Chat Completions. Each agent step and each chat turn is a separate request with its own trace, cost, and request_id in Dashboard → Logs, which is the billing record: AnythingLLM computes its own token metrics for streamed replies in the app unless GENERIC_OPEN_AI_REPORT_USAGE=true asks for usage in the stream. Every workspace uses the one system-level key, so split spend by instance, or by reading the traces, rather than by workspace. ## Which model ID should AnythingLLM send? Copy the exact model ID from /models, preserving case, hyphens, and version suffixes; do not substitute a display name. Open its detail page and match the supported API endpoints, context window, and capabilities such as tool calling to the provider and features selected in AnythingLLM. A catalog listing does not mean the client can use every feature of that model. Give each tool a dedicated API key with a maxSpend cap. ## Which API protocol is AnythingLLM using? OpenAI-compatible describes an interface format; it does not make Chat Completions (/v1/chat/completions), Responses (/v1/responses), and Anthropic Messages (/v1/messages) interchangeable. Check the installed client version, provider configuration, and actual request path against the model detail page and API compatibility fact sheet. A successful plain-text chat does not establish support for hosted tools, conversation state, or file-editing features. ## Verify the AnythingLLM call in your request trace Send a simple text request from AnythingLLM, then match its trace in Dashboard → Logs by time, model, and request_id: tokens, cost, latency, and status. Next, test streaming, tool calls, and multi-turn history separately. For failures, retain the actual request path, full error message, and request_id. If there is no matching log, check client configuration and connectivity before attributing the error to the gateway or upstream. ## FAQ ### Does Base URL take https://api.router.one/v1 or the host root? The /v1 form. AnythingLLM hands the value unchanged to the openai SDK, which appends /chat/completions, and the official docker/.env.example shows a base path ending in /v1 as well. With the host root, requests go to /chat/completions and the gateway answers 404 with a not_found error whose message says the base URL must end in /v1. The form's placeholder shows a bare host, but the value you enter is used as typed. Do not append /chat/completions in either place. ### Chat works, but uploading a document fails with an embedding error. Why? The embedder is a separate setting from the LLM, and Router One serves no embeddings endpoint, so an embedder pointed at the gateway fails on upload while chat keeps working. Open Settings → Embedder and keep the built-in embedder (EMBEDDING_ENGINE='native'), or pick any embedding provider AnythingLLM supports; the Generic OpenAI embedder option and EMBEDDING_BASE_PATH must not point at Router One. Vectors are stored per document, so changing the embedder later means deleting and re-embedding every upload; decide before you build a library. ### Can each workspace use a different model, or a different key? A different model, yes. The official docs describe a System LLM, a Workspace LLM that overrides it only inside that workspace, and an Agent LLM for @agent sessions. In Workspace Settings → Chat Settings, set Workspace LLM Provider to Generic OpenAI and pick a catalog ID under Workspace Chat model (empty inherits the system choice); Agent Configuration does the same for agent sessions. A different key, no: the provider reads Base URL and API Key from the system settings, so every workspace bills to the same Router One key. For separate budgets, run separate instances with their own keys and maxSpend caps. ### Which models can AnythingLLM use through the gateway? Choose a current catalog model that supports both the endpoint and the features AnythingLLM uses. Check /models and the model detail page for the exact ID, current rates, and capabilities; a family name such as GPT or Claude is not a compatibility guarantee. Seeing a model in the picker confirms discovery, so verify an actual request too. ### Models are listed, but requests fail with 400 or 404. What should I check? Record the actual request path and error message, then check the exact model ID. A 400 can indicate invalid parameters, unsupported tools, or a model/endpoint mismatch; a 404 can indicate an incorrect path or missing resource, so it does not by itself establish that a model was retired. If the error says must be called via, use the named endpoint or select a model supported on the current endpoint. Do not add or remove /v1 or /chat/completions across all clients indiscriminately. ### Does this work from Mainland China? Yes. The gateway is reachable from Mainland China without a VPN, and the configuration is identical to the global setup. ### How do I debug a 401/402/403/429? Match the request and error message in Dashboard → Logs. For 401, check whether the key was sent and is valid; for 402, check wallet balance and maxSpend; for 403, check key permissions and access restrictions. For 429, distinguish request/token limits from upstream throttling using the error details. Keep the request_id and follow the error-codes reference. ## See also - All integration guides: https://router.one/integrations - Debug API errors in AnythingLLM: https://router.one/llm-api-error-codes - API compatibility: endpoints and supported features: https://router.one/facts/api-compatibility.md - Responses API setup and limits: https://router.one/codex-responses-api - FastGPT setup: https://router.one/integrations/fastgpt - Cline setup: https://router.one/integrations/cline - @agent tool calling: https://router.one/llm-tool-calling - Per-key model cost tracking: https://router.one/llm-cost-tracking - AnythingLLM: OpenAI (Generic) LLM provider: https://docs.anythingllm.com/setup/llm-configuration/cloud/openai-generic - AnythingLLM: embedder configuration: https://docs.anythingllm.com/setup/embedder-configuration/overview - AnythingLLM: docker/.env.example: https://github.com/Mintplex-Labs/anything-llm/blob/master/docker/.env.example - What the gateway layer does: https://router.one/llm-api-gateway - OpenAI-compatible API: https://router.one/openai-compatible-api - API docs: https://router.one/docs - Canonical page: https://router.one/integrations/anythingllm - Models and per-model token rates: https://router.one/models (markdown: https://router.one/models.md) - Pricing: https://router.one/pricing - API docs (markdown): https://router.one/docs.md - Company facts: https://router.one/facts/company.md