The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Your agent’s cost problem may not be the model alone. To find out, measure each model request and workflow step, then compare spending with task success, quality, and latency. Token totals are useful, but they do not capture every tool, retrieval, hosting, or third-party charge—and the available documentation does not establish that unmeasured steps are always the biggest cost driver. Treat that claim as a hypothesis to test on your own workload.
Measure the whole task, not just the final model call
A user-visible answer may be the end of a run that included multiple model requests, tool calls, retrieval, retries, or delegated agent work. Looking only at the last response—or at a monthly provider bill—can hide which parts of the workflow consumed resources.
As an Amazon Associate I earn from qualifying purchases.
Start with representative completed tasks. Give each task a stable ID and preserve it across model requests, tools, retries, and handoffs. For every step, capture the model or service, request identifier where available, token usage, duration, status, and whether it was a retry or delegation. Attach the task category and outcome so you can compare like with like.
Recommended Free Tools
For OpenAI Agents SDK runs, the usage object reports request counts, input, output and total tokens, plus per-request entries. Its aggregate includes model calls that produce tool calls or handoffs. Use those run totals as a cross-check, not as a substitute for recording individual requests. A session can preserve conversation history, but usage reported for a run is independent; earlier messages may still be sent again as input on later turns. OpenAI Agents SDK usage documentation
#1 Best Overall
Use traces to see where the work happened
A trace makes the sequence of work inspectable. OpenAI tracing groups activity into sessions, turns, and spans; spans can represent model responses, tool calls, and delegated agent work, with inputs, outputs, duration, status, and recorded usage where available. This view helps answer whether a cost came from repeated model requests, an expensive tool path, or a slow external step. OpenAI observability and tracing
Do not interpret missing usage as zero. Usage information may arrive after a turn or remain unknown, so a blank or null field is an incomplete measurement, not proof that the step was free. Track status and timing even when usage is unavailable, and reconcile the gap with provider or service records where possible.
Rank #2
Build a ledger that includes more than tokens
Token counts help estimate model charges, but they are not a complete invoice. Model-call inputs and outputs can include tool definitions, conversation history, tool results, and reasoning. OpenAI’s documentation states, “Reasoning tokens are billed as output tokens.” OpenAI: Observability and usage
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Repair Windows errors before they cause bigger problems3Scan for outdated or missing drivers - takes under a minuteFor each run, keep separate entries for these cost categories:
- Model requests: provider and model, request-level input and output usage, cached or reasoning token fields when exposed, and the applicable price schedule.
- Workflow activity: tool calls, retrieval, retries, and delegated work, including duration and attempt count.
- External charges: API usage, hosting, sandbox compute, and third-party services, recorded from the source that bills for them.
Calculate model charges using the price and token categories that apply to the request, and label the result as an estimate if the captured fields omit a charge component. For example, cache-write costs may apply even where the described usage fields do not expose a separate cache-write count. Add non-model charges separately instead of treating token usage as a proxy for them.
Choose instrumentation that fits your workflow
Built-in tracing, third-party observability, and custom logging are options to assess—not a ranking established by comparative tests. Check each approach against the records you actually need:
Rank #4
- Does it represent both root and delegated agents?
- Does it capture model requests, tools, retrieval, retries, and external services?
- Can you inspect token usage and duration at the step level?
- Can you manually assign costs to non-LLM work?
- Can you connect each run to a task category and outcome?
- Can you reconcile its estimates with provider billing records?
For example, LangSmith documents automatic cost calculation for supported LLM integrations and manual cost assignment to other run types, including tools and retrieval. LangSmith cost tracking documentation An OpenAI Cookbook example shows Langfuse tracing model and tool spans and monitoring approximate cost based on token use and duration. That example demonstrates an integration, not that one observability product is more accurate or capable than another. OpenAI Cookbook: Agents SDK costs
Compare spend with completion, quality, and latency
A cheaper run is not necessarily a better run if it fails more often or produces worse results. Add an outcome or evaluator result to each task record, then compare cost, completion, quality, and latency across task types and workflow versions.
Cost per successful task can be a useful team metric, but it is not a universal formula prescribed by the cited tools. Define what counts as success for your product, include failed-run costs in the analysis, and keep the underlying cost and outcome fields visible so the aggregate does not conceal trade-offs.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Find the steps worth changing
- Choose a representative sample. Include common task types as well as costly or failed runs; an average from one narrow workflow may mislead.
- Assign a stable task ID. Carry it through every request, tool call, retry, and handoff.
- Record each model request. Capture provider and model identifiers, request IDs, and returned usage fields; compare request-level records with SDK run totals.
- Trace non-model steps. Record tool, retrieval, and delegated activity with start and end times, status, attempt information, and billable external usage where available.
- Calculate and label costs. Apply the relevant provider prices to captured token categories, mark incomplete estimates, and add tool, hosting, sandbox, and third-party charges separately.
- Attach outcomes. Record task category, completion, and a quality or evaluator result so cost can be assessed alongside performance.
- Inspect outliers. Review high-cost and failed traces for repeated calls, oversized returned context, unnecessary delegation, or steps whose cost is not justified by the outcome. Treat each as a diagnostic possibility to verify, not an assumed cause.
Reconcile your trace with provider billing
Trace-level estimates explain workflow behavior; provider usage and billing records show charges from the provider’s perspective. Compare them regularly, accounting for any fields your trace omits. OpenAI’s Usage Dashboard does not combine costs across separate organizations, so teams needing consolidated reporting should use a consistent project and account structure or perform custom analysis. OpenAI Usage Dashboard documentation
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
