# Connect RAGFlow to Router One through the OpenAI-API-Compatible provider

> Markdown mirror of https://router.one/integrations/ragflow for AI assistants and crawlers. Router One is an OpenAI-compatible LLM API gateway.
> Last updated: 2026-09-20

RAGFlow is InfiniFlow's open-source RAG engine: DeepDoc parses documents into chunked datasets, and chat assistants, search and an Agent canvas answer over them. Each of those features needs an LLM, and RAGFlow's OpenAI-API-Compatible provider sends Chat Completions requests, so one Router One key fills the LLM slot, and the VLM slot when the model reads images, with any chat model in the catalog and a cost trace per request in Dashboard → Logs. The Embedding, Rerank, ASR and TTS slots stay on another provider or a local model, because Router One serves chat only. This guide is written against RAGFlow v0.27.2, whose model settings were rebuilt around provider instances in v0.27.0. It covers the instance form, the per-model Max tokens and Tool call settings, Set default models, and which dataset and chat options multiply model requests.

## Start RAGFlow with Docker and create a dedicated key

The official quickstart targets x86 hosts with at least 4 cores, 16 GB of RAM and 50 GB of disk, Docker 24.0.0 or later and Docker Compose v2.26.1 or later. Elasticsearch needs vm.max_map_count at 262144 or higher, so set it before the first start and add it to /etc/sysctl.conf to keep it across reboots. Clone the repository, check out the release tag and start the stack from the docker directory; the commands below are the quickstart's own. Wait until docker logs shows the RAGFlow banner, then open http://IP_OF_YOUR_MACHINE, since the web UI listens on port 80 by default. Router One is a public HTTPS endpoint, so nothing like host.docker.internal applies here: the RAGFlow container only needs outbound HTTPS to api.router.one. In Router One, create a key for this deployment and give it a maxSpend cap, because RAGFlow sends every parsing task, chat turn and agent step with the one key stored on the provider instance.

`terminal`

```bash
# Elasticsearch needs this; persist it in /etc/sysctl.conf
sudo sysctl -w vm.max_map_count=262144
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/docker
git checkout -f v0.27.2
docker compose -f docker-compose.yml up -d
# Wait for the RAGFlow banner before logging in
docker logs -f docker-ragflow-cpu-1
```

## Configure RAGFlow to use the Router One base URL

Open User settings → Model providers, pick OpenAI-API-Compatible under Available models, and fill in the new-instance card; the block below lists its fields in order. Instance name is a label for this set of credentials, and any value works except 'default', which the backend rejects. Base URL takes https://api.router.one/v1: the provider passes it to the official openai Python SDK, which appends /chat/completions, so every chat request is POST /v1/chat/completions. From v0.27.0 a helper named ensure_v1 adds /v1 when the URL carries no version segment, while earlier releases use the value exactly as typed, so typing /v1 is correct on every version; never paste the full /chat/completions path. API Key is the Router One key. Under Models, List models asks the gateway's /v1/models for the catalog, and the + button opens Add custom model for a single ID. Model name must be the exact catalog ID: RAGFlow builds its internal reference as model@instance@provider and splits it only on the @ sign, and the provider class strips nothing but a ___ suffix, so an ID with a slash such as anthropic/claude-sonnet-5 goes out in the model field as typed. Model type is Chat. Max tokens is a context budget in the source, even though the setup guide words it as a generation limit: RAGFlow stores it as the model's max_length, packs retrieved chunks up to 97% of it, trims the message list to 95% of it, falls back to 8192 when it is 0, and removes max_tokens from the outgoing request, so copy the context window from the model detail page. Switch on Tool call only when the model detail page lists tool calling. Save sends a real test chat (a streamed 'Hi') and keeps the instance once one model answers; then open Set default models and choose the LLM:

`ragflow-model-provider`

```text
# RAGFlow v0.27+ → 用户设置 → 模型提供商 → 可选模型 → OpenAI-API-Compatible
# RAGFlow v0.27+ → User settings → Model providers → Available models → OpenAI-API-Compatible
实例名称（Instance name）:     router-one   # 不能叫 default / anything except 'default'
基础 URL（Base URL）:          https://api.router.one/v1
API Key:                       sk-your-router-one-key

# 模型（Models）→ 模型列表（List models），或点 + 添加自定义模型（Add custom model）
模型名称（Model name）:        <exact-model-id-from-/models>
模型类型（Model type）:        Chat   # 支持图片输入时可加选 VLM / add VLM only for image input
最大 Token 数（Max tokens）:   <context-window-from-the-model-detail-page>
模型特性（Model features）:    Tool call = on   # 仅当模型详情页列出工具调用 / only if the model page lists tool calling

# 保存（Save）→ 设置默认模型（Set default models）
LLM:                           <exact-model-id-from-/models>
Embedding / Rerank / ASR / TTS: 其他提供商或本地模型 / another provider or a local model
```

## Which RAGFlow field sends which request

Everything Router One needs sits in one provider instance plus one model entry per catalog ID. On a new instance RAGFlow pre-adds whatever List models returns, tagged by name hints: every Router One ID arrives as type Chat, any image-generation ID in the list included, with Tool call off, and with Max tokens at 8192 unless the list response carries a context field RAGFlow recognizes. Remove the rows you will not use with the minus button instead of keeping the whole list, and open Edit model on each row you keep:

| RAGFlow field or slot | What it sends to Router One | What to enter or verify |
| --- | --- | --- |
| Instance → Base URL | POST /v1/chat/completions for every chat request; GET /v1/models for List models | https://api.router.one/v1; v0.27.0 and later add a missing /v1, older releases do not |
| Instance → API Key | Authorization: Bearer on every request from this instance | A Router One key created for this deployment, with maxSpend; the save-time test appears in Dashboard → Logs |
| Add custom model → Model name | The model field, sent as typed | The exact ID from /models, vendor prefix included; its detail page must list POST /v1/chat/completions |
| Model type | Chat: Chat Completions. VLM: Chat Completions with image_url parts | Chat for every model; add VLM only when the detail page lists image input. Never Embedding or Rerank |
| Max tokens | Nothing on the wire; RAGFlow's own context budget for history and retrieved chunks | The context window from the model detail page; 0 falls back to 8192 |
| Model features → Tool call | The tools array on Agent component requests | On only when the detail page lists tool calling; when off, RAGFlow logs a warning and sends no tools |
| Set default models → LLM | Parsing-time requests: Auto-keyword, Auto-question and Auto metadata, once per chunk | A Router One chat model; every such request is one row in Logs |
| Set default models → Embedding, Rerank, ASR, TTS | Nothing: Router One has no embeddings, rerank or audio endpoint | Another provider or a local model; RAGFlow asks for a default LLM and a default Embedding model at minimum |

## Where the embedding, rerank and speech models come from

RAGFlow's setup guide asks for two defaults at minimum, an LLM and an Embedding model, and Router One can only be the first. Since v0.22.0 RAGFlow ships only the slim image, and docker/.env states that v0.22+ images include no embedding models, so choose one of three sources. The Compose stack carries an optional text-embeddings-inference service: in docker/.env, uncomment the COMPOSE_PROFILES line that adds tei-cpu (or tei-gpu), pick TEI_MODEL (the file lists BAAI/bge-m3 and BAAI/bge-small-en-v1.5 among its options, with their memory needs), and set HF_ENDPOINT=https://hf-mirror.com if the host has limited access to huggingface.co. A local runtime works too: the official local-model guide covers Ollama and Xinference, and uses bge-m3 on Ollama as its embedding example. Or use any embedding provider from RAGFlow's supported-models list. The quickstart warns that a dataset's embedding model cannot be changed once a file has been parsed with it, so decide before you build a library. Inside the Router One instance, never give a model the Embedding or Rerank type: those types post to /v1/embeddings and /v1/rerank, which the gateway does not serve, and ASR and TTS models likewise have to come from a provider that serves audio. Retrieval, reranking and the document store (Elasticsearch or Infinity) run inside RAGFlow; only the assembled prompt reaches Router One.

## Budget a deployment: which actions multiply model requests

Parsing is where request counts grow fastest. With Auto-keyword, Auto-question or Auto metadata enabled in a dataset's configuration, the task executor calls the LLM once per chunk for each enabled option, so a document that splits into 2,000 chunks with two options on is about 4,000 requests; RAGFlow caches each result by model and chunk text. Knowledge compilation, which replaced the older Knowledge graph (GraphRAG) and RAPTOR options in v0.27.0 with Graph, Tree, Wiki, PageIndex, MindMap and Timeline templates, runs on each template's Default extraction model and, per the runtime configuration docs, allows up to 20 concurrent LLM calls per task by default (WIKI_MAP_LLM_POOL_SIZE, DOC_STRUCTURE_LLM_POOL_SIZE). In a chat assistant each turn is one answer request, plus one more for each of Multi-turn optimization (from the second question on), Keyword analysis and Cross-language search, all three of which sit in the assistant's chat settings. On the Agent canvas every Agent component run is at least one request and every tool round another, up to Max reflection rounds (5 by default in the source), and Categorize calls the model as well. RAGFlow also retries on its own: the chat class retries rate-limit and server errors up to LLM_MAX_RETRIES times (default 5), and each attempt that reaches the gateway is a separate trace and charge. Verify all models sends one real test request per listed model. Give the deployment a dedicated key with maxSpend, so a runaway parsing job stops at HTTP 402 while the wallet and your other keys stay untouched; since a provider can hold several instances, a second OpenAI-API-Compatible instance with its own key separates parsing spend from chat spend. Dashboard → Logs is the billing record, with model, tokens, cost, latency, status and request_id per request; RAGFlow's own token tally for streamed replies is estimated locally whenever a chunk carries no usage figure.

## Which model ID should RAGFlow send?

Copy the exact model ID from /models, preserving case, hyphens, and version suffixes; do not substitute a display name. Open its detail page and match the supported API endpoints, context window, and capabilities such as tool calling to the provider and features selected in RAGFlow. A catalog listing does not mean the client can use every feature of that model. Give each tool a dedicated API key with a maxSpend cap.

## Which API protocol is RAGFlow using?

OpenAI-compatible describes an interface format; it does not make Chat Completions (/v1/chat/completions), Responses (/v1/responses), and Anthropic Messages (/v1/messages) interchangeable. Check the installed client version, provider configuration, and actual request path against the model detail page and API compatibility fact sheet. A successful plain-text chat does not establish support for hosted tools, conversation state, or file-editing features.

## Verify the RAGFlow call in your request trace

Send a simple text request from RAGFlow, then match its trace in Dashboard → Logs by time, model, and request_id: tokens, cost, latency, and status. Next, test streaming, tool calls, and multi-turn history separately. For failures, retain the actual request path, full error message, and request_id. If there is no matching log, check client configuration and connectivity before attributing the error to the gateway or upstream.

## FAQ

### Saving fails with 'Fail to access model(OpenAI-API-Compatible/…).No valid response received'. What does it mean?

That message is RAGFlow's generic verdict, not the gateway's answer. On Save, and on each row's verify button, RAGFlow streams a one-word test chat to the model; when the request errors it drops the detail and reports Fail to access model(OpenAI-API-Compatible/your-model-id).No valid response received, or Timeout accessing model(…) when nothing arrives within 10 seconds, the verification default of LLM_TIMEOUT_SECONDS. The real cause is in the server log: run docker logs -f docker-ragflow-cpu-1 and look for the line containing async base giving up, which carries the HTTP status and the gateway's message. A 401 AUTH_INVALID_API_KEY means the key was pasted wrong or revoked. A 404 not_found whose message tells you to fix the base URL means the path is off: a full /v1/chat/completions URL in Base URL makes the SDK append /chat/completions a second time, and releases before v0.27.0 do not add a missing /v1 for you. A 400 whose message says must be called via means the ID belongs on another endpoint, which is what happens to an image-generation ID that List models tagged as Chat; for any other 400, compare the Model name with /models character for character and remember that the test request carries temperature 0.9. If Dashboard → Logs shows no row for the attempt, check the original response and the RAGFlow server log before concluding that the request never reached the gateway. Records awaiting pricing can be temporarily hidden. For connection errors, verify outbound HTTPS and DNS from the RAGFlow host. Inside a chat assistant the same failures appear in the answer as a message that starts with ERROR, followed by RAGFlow's error class and the gateway's text.

### The model test passes, but parsing a document fails at the embedding step. Can the embedding model use Router One too?

No. Router One serves chat models on /v1/chat/completions and has no embeddings, rerank or audio endpoint, while RAGFlow embeds every chunk during parsing and every question during retrieval. If Set default models has no Embedding entry, or the entry points at a model typed Embedding inside the Router One instance, the LLM test passes and parsing fails. Set the default Embedding model to the Compose stack's optional TEI service (the tei-cpu or tei-gpu profile in docker/.env), to an Ollama or Xinference model such as bge-m3, or to another embedding provider, and leave Rerank empty or on a provider that serves one. Pick it before you parse anything: RAGFlow does not let a dataset change its embedding model after a file has been parsed with it.

### I run RAGFlow v0.26 or older and see one Add LLM dialog instead of instances. What changes?

The values are the same; only the form differs. Releases before v0.27.0 open a single dialog for OpenAI-API-Compatible with Model type, Model name, Base url, API-Key and Max tokens among its fields, plus the switches Enable tool call and Does it support Vision?. Choose chat as the Model type, enter the exact catalog ID as Model name, https://api.router.one/v1 as Base url, and the context window from the model detail page as Max tokens; RAGFlow's own UI string for that field describes it as the maximum context size of the model. Two details differ. The base URL is used exactly as typed, so a missing /v1 sends the test to /chat/completions and the gateway answers 404 not_found. And RAGFlow stores the model under an internal name ending in ___OpenAI-API; the provider class removes that suffix before every request, so the gateway still receives the plain catalog ID. The release notes date the revamped model provider system to v0.27.0, released on August 19, 2026.

### Which models can RAGFlow use through the gateway?

Choose a current catalog model that supports both the endpoint and the features RAGFlow uses. Check /models and the model detail page for the exact ID, current rates, and capabilities; a family name such as GPT or Claude is not a compatibility guarantee. Seeing a model in the picker confirms discovery, so verify an actual request too.

### Models are listed, but requests fail with 400 or 404. What should I check?

Record the actual request path and error message, then check the exact model ID. A 400 can indicate invalid parameters, unsupported tools, or a model/endpoint mismatch; a 404 can indicate an incorrect path or missing resource, so it does not by itself establish that a model was retired. If the error says must be called via, use the named endpoint or select a model supported on the current endpoint. Do not add or remove /v1 or /chat/completions across all clients indiscriminately.

### Does this work from Mainland China?

Yes. The gateway is reachable from Mainland China without a VPN, and the configuration is identical to the global setup.

### How do I debug a 401/402/403/429?

Match the request and error message in Dashboard → Logs. For 401, check whether the key was sent and is valid; for 402, check wallet balance and maxSpend; for 403, check key permissions and access restrictions. For 429, distinguish request/token limits from upstream throttling using the error details. Keep the request_id and follow the error-codes reference.

## See also

- All integration guides: https://router.one/integrations
- Debug API errors in RAGFlow: https://router.one/llm-api-error-codes
- API compatibility: endpoints and supported features: https://router.one/facts/api-compatibility.md
- Responses API setup and limits: https://router.one/codex-responses-api
- Agno setup: https://router.one/integrations/agno
- MaxKB setup: https://router.one/integrations/maxkb
- FastGPT setup: the sibling knowledge-base platform: https://router.one/integrations/fastgpt
- Connection and /v1 path troubleshooting: https://router.one/api-connection-troubleshooting
- Tool-calling requirements for Agent components: https://router.one/llm-tool-calling
- RAGFlow: Configure Model API Key: https://ragflow.io/docs/llm_api_key_setup
- RAGFlow: Quickstart: https://ragflow.io/docs/
- RAGFlow: docker/.env (embedding service and image options): https://github.com/infiniflow/ragflow/blob/main/docker/.env
- What the gateway layer does: https://router.one/llm-api-gateway
- OpenAI-compatible API: https://router.one/openai-compatible-api
- API docs: https://router.one/docs
- Canonical page: https://router.one/integrations/ragflow
- Models and per-model token rates: https://router.one/models (markdown: https://router.one/models.md)
- Pricing: https://router.one/pricing
- API docs (markdown): https://router.one/docs.md
- Company facts: https://router.one/facts/company.md