Does streaming cost more?

No. Token accounting is identical to a non-streamed request for the same content — streaming only changes how the response is delivered. The per-request cost trace shows the same fields either way.

Nothing arrives for several seconds — is the request stuck?

Usually not: the model is processing your prompt before emitting the first token, and long prompts or reasoning-heavy models stretch that phase. Resist adding a short timeout — check the request's end-to-end latency in Dashboard → Logs before concluding anything is wrong.

Can tool calls be streamed?

Yes — tool-call fragments arrive incrementally in the delta chunks: accumulate the id, name, and argument pieces until the stream finishes with finish_reason tool_calls, then run the normal tool loop. See the LLM tool calling guide.

What happens if the upstream fails mid-stream?

Retryable upstream failures can move to another eligible route before generation is underway; once you are receiving tokens, a hard upstream failure surfaces as a terminated stream that your client should handle. The automatic fallback page explains which failures are retryable.

Do all models stream?

Router One follows the OpenAI Chat Completions contract, including streaming, with support varying by underlying model. Check the model's entry in the /models catalog, and the docs for current per-feature details.

Streaming LLM responses over one endpoint

Streaming turns a chat completion into Server-Sent Events: tokens render as the model produces them, so users read while the model writes. Router One follows the OpenAI Chat Completions contract — set stream to true and the same code path streams GPT, Claude, Gemini, or DeepSeek models, with support varying by model. This page covers the wire format, the client patterns that hold up in production, and the two operational questions streaming always raises: timeouts and cost accounting.

Get your API key

Turn it on

One field. The -N flag keeps curl from buffering, so chunks print as they arrive. Pick any model ID from the /models catalog:

stream.sh

curl -N https://api.router.one/v1/chat/completions \
  -H "Authorization: Bearer sk-your-router-one-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<model-id-from-/models>",
    "messages": [{"role": "user", "content": "Write a haiku"}],
    "stream": true
  }'

What comes over the wire

The response is a text/event-stream: each SSE line carries a chat.completion.chunk object whose choices[].delta holds a fragment — the first chunk typically carries the role, later ones carry content pieces, and the stream ends with a literal [DONE]:

wire-format.txt

data: {"object":"chat.completion.chunk","choices":[{"delta":{"role":"assistant"},"index":0}]}

data: {"object":"chat.completion.chunk","choices":[{"delta":{"content":"Autumn"},"index":0}]}

data: {"object":"chat.completion.chunk","choices":[{"delta":{"content":" moon"},"index":0}]}

data: {"object":"chat.completion.chunk","choices":[{"delta":{},"finish_reason":"stop","index":0}]}

data: [DONE]

Client patterns

The official SDKs hide the SSE parsing — you iterate chunks and concatenate deltas. The same loop works for every model family on the endpoint:

stream.py

from openai import OpenAI

client = OpenAI(
    base_url="https://api.router.one/v1",
    api_key="sk-your-router-one-key",
)

stream = client.chat.completions.create(
    model="<model-id-from-/models>",
    messages=[{"role": "user", "content": "Write a haiku"}],
    stream=True,
)
for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)

Timeouts, cancellation, and long generations

Time-to-first-token and total duration are different budgets. A large prompt or a reasoning-heavy model can think for a while before the first chunk — that silence is normal, not a hang.
Do not set aggressive client timeouts on generation endpoints; they convert healthy long generations into client-side failures. If you must bound something, bound idle time between chunks, generously.
Cancelling is just closing the HTTP connection — no special API. Wire your UI's stop button to abort the request.
Judge latency from the trace, not by feel: the request row in Dashboard → Logs records end-to-end timing, so you can tell a slow network hop from a slow generation.

Streamed requests are still fully traced

Streaming changes the delivery, not the accounting. Every streamed request lands in Dashboard → Logs as one row with the final model, tokens, cost, latency, and status — the same per-request observability as non-streamed calls, and the same per-key budgets apply. If a stream fails mid-flight, the row's status tells you whether it ever started generating.

FAQ

Does streaming cost more?: No. Token accounting is identical to a non-streamed request for the same content — streaming only changes how the response is delivered. The per-request cost trace shows the same fields either way.
Nothing arrives for several seconds — is the request stuck?: Usually not: the model is processing your prompt before emitting the first token, and long prompts or reasoning-heavy models stretch that phase. Resist adding a short timeout — check the request's end-to-end latency in Dashboard → Logs before concluding anything is wrong.
Can tool calls be streamed?: Yes — tool-call fragments arrive incrementally in the delta chunks: accumulate the id, name, and argument pieces until the stream finishes with finish_reason tool_calls, then run the normal tool loop. See the LLM tool calling guide.
What happens if the upstream fails mid-stream?: Retryable upstream failures can move to another eligible route before generation is underway; once you are receiving tokens, a hard upstream failure surfaces as a terminated stream that your client should handle. The automatic fallback page explains which failures are retryable.
Do all models stream?: Router One follows the OpenAI Chat Completions contract, including streaming, with support varying by underlying model. Check the model's entry in the /models catalog, and the docs for current per-feature details.

Get your API key