LLM observability: one trace per request
Router One records model-call usage in Dashboard → Logs. Filter by model, API key, status and date range, then open a row for its request ID, the key that sent it, tokens, total time (plus time to first token on streamed requests that produced output), status and any sanitized error. Check the stream's final event separately from its HTTP status, and use the recorded settlement to confirm its charge. Support can correlate the request ID with internal routing attempts.
What can you inspect in a request log?
| Signal | Router One trace | How to interpret it |
|---|---|---|
| Request ID & model | Open the row to copy requestId and inspect the recorded model | Match the ID with the same call in your application's logs |
| Input / output tokens | Recorded input/output counts and cache-read tokens when available | Usage does not prove the client received a complete answer |
| Cost per request | Recorded totalCost after settlement; pending means the amount is unknown | The settled total already includes applicable pricing multipliers |
| Latency | Total time on each request record, plus time to first token (TTFT) on streamed requests that produced output; a missing total displays as a dash | Both are timed from when the gateway starts executing the request, so TTFT includes any failed attempt before failover; the gap to your client's first visible token is your network, your client and the gateway's request intake |
| Routing investigation | A request ID for support correlation | Provider names and the attempt chain are not displayed in customer Logs |
| Status code & errors | Status code and sanitized error text when available | HTTP 200 alone cannot prove a streamed answer completed |
| Spend ceiling | Per-key maxSpend stops a runaway loop | Set a separate loop limit in your application too |
Where to look
Dashboard → Logs
Filter by model, API key, status and date range, then open the matching row for its request ID, usage, status and settlement details; the API key column (and the request detail) names the key that sent each request — web-console calls are labeled as such. Logs (current filters) and the wallet ledger (chosen range) export to CSV. A record may not appear immediately while pricing is pending. For attribution below key level (per end customer or run), keep your own labels with request IDs in your application logs.
Dashboard → Usage
Review request, token and spend trends plus per-model and capability breakdowns. Usage also ranks your API keys by spend, tokens and requests for the chosen range, and each key links to its own logs. Inspect the individual Logs row for the recorded charge and any supplied discount details.
Budgets & default rate limits
Each API key carries its own maxSpend; its rateLimit and tokenLimitTpm run at platform defaults and can be raised on request. A runaway loop hits its own ceiling and stops instead of draining the whole balance — spend control without a governance suite.
Example: a streamed request awaiting settlement
Synthetic excerpt using customer Logs fields. Values are illustrative, not a production trace or price quote. If a row is marked pending, its final cost is unknown; records awaiting pricing may temporarily be absent from the list.
{
"requestId": "demo_req_7f3a",
"model": "example-chat-model",
"inputTokens": 512,
"outputTokens": 200,
"firstTokenLatencyMs": 640,
"latencyMs": 1180,
"statusCode": 200,
"totalCost": null,
"billingPricing": {
"status": "pending"
}
}Request-data handling
The customer trace contains request metadata for debugging and billing review. Router One does not retain prompt or model-response bodies from direct API calls. It records request metadata for billing, usage tracking, and troubleshooting. Playground saves conversation history so users can revisit and continue their chats.
Read the data retention policyFAQ
Does HTTP 200 mean my streamed answer succeeded?
No. HTTP 200 can indicate that the stream opened before generation failed or became incomplete. For Chat Completions, inspect finish_reason and any stream error; stop, length and tool_calls mean different things. For Responses, examine response.completed, response.failed or response.incomplete. A closed connection or a dashboard success badge alone is insufficient.
Does Pending settlement mean this request was free?
No. Pending settlement means the final charge is unknown. The UI does not substitute $0 or undiscounted token subtotals. Check the row again later; the settled total already includes applicable discounts. Any displayed default or model-specific discount, plan multiplier and effective multiplier explain that total rather than being another discount to apply.
What evidence should I collect for a slow or failed request?
Keep the request ID, timestamp with timezone, endpoint, exact model ID, SDK version, timeout settings, HTTP status and final error or finish reason. Record when the first visible token arrived in your client and whether you cancelled, then compare it with the time to first token (TTFT) in Logs: the gap between the two is your network, your client and the gateway's request intake; a long TTFT itself points at queueing, a failed attempt before failover, or the model's own wait. Compare similar models, endpoints and prompt sizes across multiple calls; a single duration cannot identify the cause. Remove API keys and sensitive input/output before sharing.
What exactly is in a trace?
Customer Logs shows the recorded model, request ID, the API key that sent it, input/output tokens, available cache-read tokens, settled cost or a pending label, total time (plus time to first token, TTFT, for streamed requests that produced output), status code and any sanitized error. Pricing details appear when supplied. The logs CSV export carries a TTFT (ms) column next to Latency (ms). Provider names and intermediate attempt chains are not displayed there.
Does Router One store my prompts and responses?
Router One does not retain prompt or model-response bodies from direct API calls. It records request metadata for billing, usage tracking, and troubleshooting. Playground saves conversation history so users can revisit and continue their chats.
How do I stop a runaway loop from draining my balance?
Give the API key a maxSpend; its rateLimit and tokenLimitTpm apply at platform defaults and can be raised on request. When a loop hits the key's spend ceiling or rate limit, requests on that key stop instead of consuming the whole wallet. Budgets are per key, not per project.
Is this an enterprise audit or compliance platform?
No. This is request-level observability and spend control for developers and small teams. It is not an enterprise audit, compliance, or RBAC system. There is no organization/role structure — limits and budgets attach to API keys.
What does the trace show when a provider fails?
Customer Logs records the resulting model-call usage, status and any sanitized error. It does not show provider names or the fallback chain. Support can use the request ID to investigate intermediate attempts in operational logs; retain the client error and terminal event too.
Related
- LLM streaming and terminal events
- API errors and timeout diagnosis
- API connection troubleshooting
- Pricing methodology
- LLM API gateway
- Automatic LLM fallback
- OpenAI-compatible API
- LLM cost tracking
- Reseller key setup
- Per-workload cost attribution
- Data retention
- Helicone alternative
See every request, control every dollar
Get your API key