# Tool calling with one request shape for every model

> Markdown mirror of https://router.one/llm-tool-calling for AI assistants and crawlers. Router One is an OpenAI-compatible LLM API gateway.
> Last updated: 2026-09-05

Tool calling (function calling) lets a model return a structured request to run one of your functions instead of prose. Every major family supports it, but each official API dresses it differently — which is exactly the integration tax a gateway removes. Router One follows the OpenAI Chat Completions contract, so the same tools array and the same response-handling loop work across GPT, Claude, Gemini, and Grok models; support varies by model, so check the catalog entry for the model you target.

## Declare tools in the request

Tools are JSON Schema function declarations attached to a normal chat completion. Pick any tool-capable model ID from the /models catalog:

`request.sh`

```bash
curl https://api.router.one/v1/chat/completions \
  -H "Authorization: Bearer sk-your-router-one-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<model-id-from-/models>",
    "messages": [{"role": "user", "content": "Weather in Shanghai?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get current weather for a city",
        "parameters": {
          "type": "object",
          "properties": {"city": {"type": "string"}},
          "required": ["city"]
        }
      }
    }]
  }'
```

## The tool-call loop

Handling a tool call is a four-step conversation, identical regardless of which model family serves the request:

### 1. The model asks for a tool

Instead of content, the response carries finish_reason "tool_calls" and a tool_calls array — each entry has an id, the function name, and the arguments as a JSON string.

### 2. You run the function

Parse the arguments string, validate it against your schema, and execute the real function. The model never runs anything itself — it only proposes the call.

### 3. You return the result

Append the assistant message you received, then a message with role "tool", the matching tool_call_id, and the function result as its content.

### 4. The model finishes the answer

Send the extended message list back to the same endpoint. The model folds the tool result into its final user-facing reply — or requests another tool, in which case the loop repeats.

`tool_loop.py`

```python
from openai import OpenAI
import json

client = OpenAI(
    base_url="https://api.router.one/v1",
    api_key="sk-your-router-one-key",
)

resp = client.chat.completions.create(
    model="<model-id-from-/models>", messages=messages, tools=tools
)
msg = resp.choices[0].message

if resp.choices[0].finish_reason == "tool_calls":
    messages.append(msg)
    for call in msg.tool_calls:
        result = run_tool(call.function.name,
                          json.loads(call.function.arguments))
        messages.append({
            "role": "tool",
            "tool_call_id": call.id,
            "content": json.dumps(result),
        })
    final = client.chat.completions.create(
        model="<model-id-from-/models>", messages=messages, tools=tools
    )
```

## What varies by model

- Whether tool calling is supported at all — check the capability flags on the model's catalog entry before you build on it.
- Parallel tool calls: some models return several tool_calls entries in one response, others one at a time. Loop over the array instead of reading index 0.
- Schema strictness: models differ in how faithfully arguments follow your JSON Schema. Validate before executing, always.
- Eagerness: some models call tools at any opportunity, others prefer answering in text. Sharper function descriptions and the tool_choice parameter steer both kinds.
- Server-side tools on /v1/messages: on the Anthropic-compatible /v1/messages endpoint the web_search and code_execution server tools (including code_execution's bash and text-editor sub-tools) are accepted and metered at the model's standard token rate; web_fetch and mcp_servers are rejected with a 400 before any model is called. The client-side function tools on this page ride on /v1/chat/completions for every tool-capable model.

## Debugging tool calls on the gateway

Two failure modes dominate. The model answers in prose instead of calling the tool: tighten the function description, name the trigger conditions, or force it with tool_choice. The arguments do not parse or fail validation: never eval them blind — validate, and on failure send the error back as the tool result so the model can correct itself. Either way, every attempt is one request row in Dashboard → Logs with tokens, cost, latency, and status, so you can see exactly what each iteration of the loop cost.

## FAQ

### Which models support tool calling?

Support varies by model. Router One follows the OpenAI Chat Completions contract, and the /models catalog marks each model's capabilities — check the entry for the model you plan to use rather than assuming family-wide support.

### Do tool schemas count against my tokens?

Yes — tool declarations travel with the prompt and are billed as input tokens, on every iteration of the loop. The per-request trace in Dashboard → Logs shows the token and cost impact, so oversized schemas show up immediately.

### Tool calling or structured outputs — which one do I want?

Tool calling is for actions: the model decides to invoke your function and you feed the result back. If you only need the reply in a fixed JSON shape with no function to run, a schema-constrained response via response_format (json_object, or json_schema with a named schema) is the simpler pattern — the gateway checks the envelope and forwards it to the model; whether the schema is enforced depends on the model. Many apps use both together. See the structured outputs guide.

### Can I stream a response that contains tool calls?

Yes. With stream enabled, tool-call fragments arrive incrementally in the delta chunks — accumulate id, name, and argument fragments until the stream finishes with finish_reason tool_calls, then run the normal loop. The LLM streaming guide covers the chunk format.

### Does the same loop really work across model families?

That is the point of the OpenAI-compatible contract: the tools array, tool_calls response, and tool-role result message keep one shape, and Router One routes the request to whichever model you named. Behavioral differences (parallel calls, strictness) still exist per model — the loop structure does not change.

## See also

- OpenAI-compatible API: https://router.one/openai-compatible-api
- LLM streaming: https://router.one/llm-streaming
- Structured outputs (JSON Schema): https://router.one/llm-structured-outputs
- OpenAI SDK setup: https://router.one/integrations/openai-sdk
- LangChain setup: https://router.one/integrations/langchain
- LlamaIndex setup: https://router.one/integrations/llamaindex
- Claude web search & code execution on /v1/messages: https://router.one/blog/claude-web-search-code-execution-api
- Error codes reference: https://router.one/llm-api-error-codes
- What the gateway layer does: https://router.one/llm-api-gateway
- API docs: https://router.one/docs
- Canonical page: https://router.one/llm-tool-calling
- Models and per-model token rates: https://router.one/models (markdown: https://router.one/models.md)
- Pricing: https://router.one/pricing
- API docs (markdown): https://router.one/docs.md
- Company facts: https://router.one/facts/company.md