# Structured outputs: JSON mode and JSON Schema on one request shape

> Markdown mirror of https://router.one/llm-structured-outputs for AI assistants and crawlers. Router One is an OpenAI-compatible LLM API gateway.
> Last updated: 2026-08-22

Structured outputs ask a model for JSON instead of prose — either any valid JSON object (json_object) or JSON that follows a schema you name (json_schema). Router One follows the OpenAI Chat Completions contract, so the response_format field rides on the same request you already send to every chat model in the catalog. The gateway checks the envelope, answers 400 before any model is called when the schema part is incomplete, and otherwise forwards response_format to the model without rewriting it. Whether the schema is enforced is up to the model — support and strictness vary, so parse and validate what comes back.

## json_object or json_schema?

Both live under response_format. Pick by how much shape you need:

### json_object — JSON mode

The model returns a syntactically valid JSON object; the keys and nesting are whatever your prompt asks for. Always tell the model in the prompt that you want JSON and describe the fields — the flag alone does not name them.

### json_schema — a named schema

You attach a JSON Schema with a name, optionally a description and strict: true. A model that honours it constrains its output to that schema; a model that does not still replies, just without the constraint. That difference is the whole reason to validate client-side.

## JSON mode request

Pick any chat model ID from the /models catalog. The system message carries the field list; response_format carries the mode:

`json_object.sh`

```bash
curl https://api.router.one/v1/chat/completions \
  -H "Authorization: Bearer sk-your-router-one-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<model-id-from-/models>",
    "messages": [
      {"role": "system", "content": "Reply with a JSON object with keys city and date."},
      {"role": "user", "content": "Extract the city and date: meeting in Shanghai on 3 September."}
    ],
    "response_format": {"type": "json_object"}
  }'
```

## JSON Schema request

The json_schema object needs a name and a schema; description and strict are optional and are forwarded with the rest:

`json_schema.sh`

```bash
curl https://api.router.one/v1/chat/completions \
  -H "Authorization: Bearer sk-your-router-one-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<model-id-from-/models>",
    "messages": [{"role": "user", "content": "Extract the city and date: meeting in Shanghai on 3 September."}],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "meeting",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {
            "city": {"type": "string"},
            "date": {"type": "string"}
          },
          "required": ["city", "date"],
          "additionalProperties": false
        }
      }
    }
  }'
```

## What the gateway checks before forwarding

With type json_schema, Router One validates the envelope and rejects an incomplete one with 400 invalid_request before any model is called — the request never reaches a model, so there is no token usage to bill. The three messages, verbatim:

| Request | 400 message | Fix |
| --- | --- | --- |
| type is json_schema but json_schema is absent | response_format.json_schema is required when response_format.type is json_schema | Add the json_schema object with name and schema |
| json_schema has no name | response_format.json_schema.name is required | Give the schema a short identifier such as meeting |
| json_schema has no schema | response_format.json_schema.schema is required | Put the JSON Schema object under schema |

- A complete envelope is forwarded as sent — name, description, strict and schema included. The gateway does not trim, rewrite or re-validate the schema body.
- The gateway does not validate the model's reply against your schema either. Enforcement is the model's job; verification is yours.

## Support and strictness vary by model

- json_object is honoured broadly: you get well-formed JSON, and the prompt decides the shape.
- json_schema and strict are honoured by a subset of models. Where a model does not apply the schema you still get a 200 and a reply — it is simply not constrained. Models served over a native, non-OpenAI protocol may honour json_object only, or not apply the schema at all.
- The /models catalog marks chat, streaming, tool calling and vision per model; it does not carry a structured-outputs flag. Send one small test request to the model you plan to use and read the reply in the trace before you build on it.
- Need one shape that holds across model families? Declare a single tool and force it with tool_choice — the arguments arrive as a JSON string following your parameters schema on tool-capable models. Validate before use either way.

## Schema dialect: $defs, $ref and candidate failover

Some model routes reject $defs and $ref inside a JSON Schema because their native schema dialect has no equivalent. Router One treats that rejection as specific to the candidate that raised it, not to the model: the request fails over to the next candidate for the same model, and the rejection is not counted against the model's health. If every candidate rejects the dialect the 400 comes back to you — inline the definitions (flatten every $ref) and the same schema goes through. The candidate that finally served the request is the one shown in Dashboard → Logs.

## Streaming and SDK usage

Set stream: true as usual. With json_schema the JSON arrives as content deltas in the normal SSE chunk format — accumulate content until the final chunk and parse once; never parse a partial object mid-stream. With the OpenAI SDK, response_format is a plain keyword argument:

`structured.py`

```bash
from openai import OpenAI
import json

client = OpenAI(
    base_url="https://api.router.one/v1",
    api_key="sk-your-router-one-key",
)

schema = {
    "type": "object",
    "properties": {"city": {"type": "string"}, "date": {"type": "string"}},
    "required": ["city", "date"],
    "additionalProperties": False,
}

resp = client.chat.completions.create(
    model="<model-id-from-/models>",
    messages=[{"role": "user", "content": "Extract the city and date: meeting in Shanghai on 3 September."}],
    response_format={
        "type": "json_schema",
        "json_schema": {"name": "meeting", "strict": True, "schema": schema},
    },
)
data = json.loads(resp.choices[0].message.content)
# validate `data` against `schema` before you use it
```

## Reading the result in Dashboard → Logs

Every attempt is one request row with model, tokens, cost, latency and a status badge; open the row for the request detail. A 400 from the gateway's own check shows status 400 with one of the messages above — fix the request. A 200 whose reply is prose or loosely shaped JSON means the model you named did not apply the schema — tighten the prompt, add a validation-and-retry step, or switch model. The trace tells you which of the two you are looking at before you touch code.

## FAQ

### What is the difference between json_object and json_schema?

json_object (JSON mode) only guarantees that the reply is a syntactically valid JSON object — the prompt decides the keys. json_schema attaches a named JSON Schema, optionally with strict: true, so a model that supports it constrains the output to that shape. Both are sent in the same response_format field on /v1/chat/completions.

### Does Router One guarantee the reply matches my schema?

No. The gateway validates the request envelope and forwards response_format to the model as sent; it does not rewrite the schema or check the model's output against it. Enforcement depends on the model you name and varies by model, so parse and validate the reply client-side before using it.

### Does the schema count toward my tokens?

Yes — the schema travels with the prompt and is billed as input tokens on every request, and the JSON reply is billed as output tokens at the model's posted rate. A request rejected with 400 by the gateway's own check never reaches a model and has no token usage. The per-request trace in Dashboard → Logs shows the token and cost impact of a large schema immediately.

### Does json_schema work with streaming?

Yes. Set stream: true and the JSON arrives as content deltas in the same SSE chunk format as any other streamed reply. Accumulate the content fragments until the stream finishes, then parse once — a partial object never parses. The LLM streaming guide covers the chunk format, timeouts and cancellation.

### Tool calling or structured outputs — which one do I want?

Structured outputs are for a reply in a fixed JSON shape with no function to run. Tool calling is for actions: the model asks to invoke your function and you feed the result back. If you need a schema-shaped reply from a model that does not honour json_schema, a single forced tool is the portable workaround — the arguments follow your parameters schema. Many apps use both.

### What does the 400 mean, and what does it cost?

A 400 invalid_request naming response_format.json_schema, .name or .schema means the envelope was incomplete and the gateway stopped it before any model call — nothing was billed. Add the missing field and resend. Any other 400 comes from the model route itself; a $defs/$ref rejection that every candidate repeats is the common one, and flattening the schema clears it.

## See also

- LLM tool calling: https://router.one/llm-tool-calling
- LLM streaming: https://router.one/llm-streaming
- OpenAI-compatible API: https://router.one/openai-compatible-api
- Error codes reference: https://router.one/llm-api-error-codes
- Chat Completions reference: https://router.one/docs/chat/createChatCompletion
- Per-request traces: https://router.one/llm-observability
- OpenAI SDK setup: https://router.one/integrations/openai-sdk
- API docs: https://router.one/docs
- Canonical page: https://router.one/llm-structured-outputs
- Models and per-model token rates: https://router.one/models (markdown: https://router.one/models.md)
- Pricing: https://router.one/pricing
- API docs (markdown): https://router.one/docs.md
- Company facts: https://router.one/facts/company.md