Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →An agent can spend tokens before it returns useful work: the request may carry system instructions, conversation history and tool definitions, and the agent may make additional model calls as it selects tools, processes results or retries. Tiered routing controls that cost by sending simpler steps to less expensive models, checking their results, and escalating only when the task warrants it. There is no established universal percentage of an agent bill attributable to pre-reasoning overhead; the right measure is the cost of a successful task, not a model’s advertised price per token.
Why can an agent cost money before it does useful work?
A model call is not necessarily just the latest user message. Microsoft Learn’s Foundry agent router documentation says its router evaluates the full request context, including system and user messages, tool definitions and conversation history. Context sent with a request can therefore contribute to usage before the agent produces an answer that a person can use.
As an Amazon Associate I earn from qualifying purchases.
There may also be more than one call in the task. An agent can ask a model what to do, invoke a tool, send the tool result back to a model, and repeat that cycle. A retry or escalation adds another request. The total depends on the agent’s request pattern, how much context is included, the models selected and the work required; the sources do not establish a typical overhead share or a universal dollar amount.
That is why a low input price per token does not automatically mean a low cost per completed task. A cheaper model can require more calls or fail a task, while a more capable model may be worthwhile for difficult synthesis. Compare total usage through successful completion, alongside quality and latency.
#1 Best Overall
How does tiered routing reduce the bill?
Tiered routing assigns work to models according to its complexity and role, rather than sending every step to the same model. Microsoft documents these as example categories, not fixed rules for every model pool:
| Work type | Typical routing choice | Reason to use it |
|---|---|---|
| Factual recall and simple follow-ups | Fast, lower-cost model | The task may not need the capabilities of a frontier model. |
| Straightforward tool orchestration | Capable mid-tier model | It can select or manage routine tool calls without defaulting to the most expensive tier. |
| Research synthesis, multi-step reasoning or complex code generation | Higher-capability model | These tasks may benefit from stronger reasoning or synthesis. |
Microsoft Learn describes mid-tier models as handling straightforward tool calls “at a fraction of frontier model cost.” That is vendor guidance, not an independent benchmark or a guarantee for a particular workload. The appropriate tier depends on your models, tasks and quality requirements.
How to use a cheap check before escalating
A low-cost first attempt is most useful when you can verify its result. For example, a structured answer can be checked against a schema, a calculation can be recomputed, or a proposed tool call can be checked against allowed parameters. These are implementation examples; choose checks that actually detect errors in your task.
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Clear out junk files and repair common Windows errors3Scan for outdated or missing drivers - takes under a minute- Classify the work. Identify whether the step is simple recall, routine tool orchestration or a task that needs deeper reasoning. Keep the policy tied to the step’s role and complexity.
- Try the lower-cost tier when the risk permits. Use it for work where a failure can be detected and safely corrected. Do not make “cheap first” an absolute rule for consequential or inherently complex tasks.
- Run a verifiable check. Accept the result only if it passes the check and meets the task’s quality requirements. A confident-sounding answer is not, by itself, proof of correctness.
- Escalate deliberately. Send the task to a stronger model when the check fails, confidence is inadequate, or the consequences and complexity justify it. Vercel’s AI Gateway guide demonstrates escalation after a cheap answer fails a verifiable check; it does not establish a universal confidence threshold.
- Set limits and observe the path. Use per-tier or per-task budgets, and record requests, token usage, retries and escalations. A budget ceiling limits runaway spend; routing aims to reduce costs for work that stays below the ceiling. Vercel describes these as complementary controls.
What should you measure to know whether routing works?
Compare the existing system with the routed version on a representative set of your own tasks. Measure the complete task rather than an isolated model call:
- Total spend or tokens per successfully completed task.
- Completion rate and task-specific quality checks.
- Latency from the user’s request to a usable result.
- Model request count, retry rate and escalation rate.
- Where available, cache-read and cache-write token usage.
Break results out by workload, especially simple tool calls versus complex synthesis. A single average can hide a quality regression on harder tasks or extra retries on supposedly easy ones. The cited guidance supports workload-specific evaluation; it does not supply a universal benchmark or pass/fail threshold.
When does caching help—and when can it fail to?
Caching can reduce the cost of repeated context, but it is not automatically a saving. DigitalOcean’s Inference Router documentation says cache writes cost more than standard uncached input and that later reuse can receive discounts. The benefit depends on whether prefixes are reused enough to offset the write cost. Switching models or providers can affect reuse, so inspect cache-read and cache-write usage rather than assuming that enabling caching lowers the bill.
DigitalOcean’s documentation page was last verified on October 6, 2026, and described its router as public preview. It said there was no extra router charge during that preview, while customers paid for the models serving requests. Preview status and terms can change; check the service’s current documentation before relying on them.
Can reasoning effort be tuned as well as the model tier?
Yes, when the model and provider expose a reasoning-effort control. Anthropic recommends matching effort to the workload and evaluating representative cases for thinking behavior, output tokens, latency and answer quality. It cautions that reducing thinking can hurt tasks that benefit from reasoning, so a lower setting is not a safe blanket cost fix. Model support and defaults vary and may change.
Anthropic’s example system-prompt guidance says: “Extended thinking adds latency and should only be used when it will meaningfully improve answer quality, typically for problems that require multistep reasoning. When in doubt, respond directly.” Treat that as Anthropic’s suggested wording, not a general standard or independent finding.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What do published cost-reduction figures actually show?
Research papers report that routing and model cascades can reduce costs in evaluated settings, but those results should not be read as a production forecast:
| Study | Reported result | How to interpret it |
|---|---|---|
| Stanford authors’ FrugalGPT (2023) | Up to 98% cost reduction | The authors reported this experimental result for methods including prompt adaptation, model approximation and model cascades. It is not a typical or guaranteed saving for an agent. |
| AgentRouter authors’ 2026 preprint | 72% cost reduction and 97.3% of frontier-only quality retained | These are reported results on the paper’s evaluated benchmarks and annotated agent trajectories, not a general production guarantee. |
Neither figure answers how much a particular agent spends before it “thinks.” To answer that for your system, inspect its actual requests and costs through successful task completion.
Recommended Free Tools
What should you check in a router or managed service?
Whether you build a router or use a managed option, compare its behavior on your workload and inspect the controls that affect cost and reliability:
- Routing policy: Does it route per request, agent step or task role? Are the rules transparent, and can hard cases escalate?
- Quality and latency: Does it handle both structured tool calls and complex synthesis to your standard?
- Spend visibility: Can you see which model handled a request, how many tokens it used, and whether it retried or escalated? Can you set budgets by tier?
- Context and caching: Does it reuse repeated prefixes, and can you distinguish cache reads from writes?
- Compatibility and operations: Which models and tools are supported in your region? What happens during provider failover, and how much maintenance do model mappings require?
Microsoft documents Cost, Balanced and Quality routing modes for Foundry Model Router, and notes that tool support can vary by model and region. DigitalOcean describes cost/latency selection and cache-aware routing. Vercel’s guide covers centralized usage monitoring and budgets. These are vendor-described capabilities, not independent comparative evaluations; verify current availability and test each option against your own quality, latency and cost requirements.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
