Halloween DealsAmazon USHalloween week deals: useful picks before month-endAmazon US: current deals, useful picks and tech finds.Check DealsWindows FixRecommendedWindows errors stealing your time? Find the fix fastScan stability, cleanup and performance issues.Fix NowHalloween DealsAmazon USOctober picks are moving - compare before checkoutAmazon US: current deals, useful picks and tech finds.See Picks×
FREEDOM251

Freedom251 › Blog › AI cost monitoring

How to Monitor LLM Token Usage and Cost by User or Feature

By Freedom251 Team · · 11 min read

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

The reliable way to measure LLM spend per customer or product feature is to create one usage record for every model invocation. Attach a stable internal user ID and feature key when the call starts, then store the provider, exact model ID, timestamp, all returned token categories, and any provider-reported cost. Aggregate those records by user, feature, model, provider, date, and environment.

Do not equate visible prompt and answer text with billed usage. Cached-input, reasoning, audio, tool, and other categories can be reported separately. When usage is missing—especially after an interrupted stream—mark it unknown and reconcile it later instead of counting it as zero.

As an Amazon Associate I earn from qualifying purchases.

What you need to measure

A model name and token total cannot answer the questions a product or finance team usually asks: Which customer generated the spend? Which workflow is expensive? Did a retry double the cost? Are estimates different from the provider invoice? Those answers require attribution at the moment each provider call is made.

Free tools Windows power users keep installed

One-click scans. No signup required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Create a generation or usage event for every invocation, including retries, fallbacks, background jobs, and secondary calls such as retrieval, classification, or tool-selection calls when they are part of the request’s cost. A practical event contains:

#1 Best Overall
Sale
TOZO A1 Wireless Earbuds Bluetooth 5.3 Light Weight in Ear Headphones | IPX5 Waterproof,2 Mics for AI Calls,Immersive Premium Sound Bass Headset with Charging Case,32 EQ via App Black
  • [Ultra-Lightweight Ear Buds Designed for Small Ears] Each earbud weighs only 3.7g and features a compact, ergonomic in-ear design made especially for small ears. Secure, low-profile, and comfortable for workouts, all-day wear or overnight sleeping.The ultra-slim profile sits flush in your ear with no hard edges poking or rubbing, so you can even sleep on your side without discomfort.
  • [Immersive Stereo Sound with TOZO OrigX Technology] TOZO OrigX tuning delivers clear vocals, balanced mids, and natural stereo sound for music, podcasts, and videos.
  • [Long Battery Life for Daily Use] Get up to 7 hours of playtime on a single charge, with up to 32 hours total using the charging case—ideal for workdays, commuting, and extended listening sessions.
  • [Bluetooth 5.3 & Stable Connection] Bluetooth 5.3 provides fast pairing, stable wireless performance, and reduced dropouts as you move around home or office.
  • [Deep Bass with Clear Vocals] High-performance drivers produce punchy bass while keeping vocals clean and detailed for everyday listening.
  • Attribution: a stable internal user_id, a feature key such as summarize, support_reply, or document_search, and an optional account, team, or tenant ID.
  • Request context: provider, exact model identifier, request or trace ID, session ID, timestamp, environment, and service or application version.
  • Usage: input and output tokens plus every additional bucket returned by the provider, such as cached-input or reasoning tokens.
  • Money: provider-reported cost when available, and a derived cost calculated from a versioned price table.
  • Integrity state: complete, incomplete, or unavailable usage, with an error or interruption reason.
  • Raw evidence: the original usage response and pricing-version reference so that a later reconciliation can reproduce the number.

Use an internal identifier rather than a display name. Keep prompts and completions out of feature labels and tags; those fields should describe business context without leaking sensitive content.

Instrument every model call at one boundary

Put a wrapper or middleware around the code that invokes each provider. The wrapper should create the event before sending the request, pass attribution through the request’s context, capture the response in both success and error paths, and close the event only when usage is known or explicitly marked unavailable.

A provider-neutral call flow

  1. Resolve context. Obtain the authenticated user’s stable ID, feature key, tenant, environment, and a request or trace ID. If a background job has no human user, use a documented service identity such as system:nightly-index rather than leaving the field blank.
  2. Record the invocation start. Generate an event ID and persist the model, provider, feature, user, and start time before the network call. This preserves evidence for timeouts and retries.
  3. Send the request. Include the provider’s exact model ID. Do not substitute a marketing family name; prices and token behavior are model-specific.
  4. Capture returned usage. Store every token category the response exposes, the provider’s request ID, finish status, and any provider-reported cost.
  5. Handle failures explicitly. Save timeout, cancellation, rate-limit, and transport errors. If the provider may have processed the request but returned no usage, label the event usage_unavailable rather than assigning zero tokens.
  6. Calculate a provisional amount. Apply the price-table version that was effective for the model, token category, date, and service tier. Keep this amount separate from any invoice-reconciled value.
  7. Roll up later. Aggregate immutable events into daily or hourly summaries by user, feature, model, provider, environment, and status.

Keep retries visible

Retries are separate provider invocations and can incur separate charges. Give each attempt its own event and link it to a common parent request ID. A dashboard can then show both the customer-facing request and the attempts that produced its cost, instead of hiding retry spend inside one ambiguous total.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Capture the provider’s real usage fields

Token accounting differs by endpoint and model. For OpenAI API responses, the documented fields differ between Chat Completions and Responses, and some endpoints expose additional cached-input or reasoning-token detail. See the OpenAI usage and cost documentation for the current field behavior.

API response family Core fields to store Additional data to preserve when returned
Chat Completions usage.prompt_tokens, usage.completion_tokens, usage.total_tokens Cached-input and reasoning-token details, finish information, request ID, and any provider cost
Responses usage.input_tokens, usage.output_tokens, usage.total_tokens Cached-input and reasoning-token details, tool-related usage, request ID, and any provider cost

Store the raw usage object as JSON as well as normalized columns. Normalized columns make rollups fast; the raw object protects you when a provider adds a category that your schema did not yet anticipate.

Rank #2
Sale
Smart Watch for Men Women Compatible with iPhone Samsung Android Phone 1.69 inch Full Touch Screen IP68 Waterproof Bluetooh Fitness Tracker Heart Rate/Sleep Monitor (Black)
  • 【IP68 Waterproof Multifunctinal Smart Watch】Torjalph fitness activity tracker smart watch is IP68 waterproof technology ,you can wear it when swimming,handwashing,it's rain-proof which means that you perform multiple exercise modes outside without worrying about sweat or rain corroding your fitness smart watch,and this watch equipped with a 230 mAh large capacity battery magnetic charging ,and this sports activity tracker watch can be used 5-7 days If fully charged.
  • 【Larger 1.83'' Full Screen Watch & Multi-dial Selection】1.83 inch full HD color touch screen smart watch for men / women provides excellent 240*284 resolution image quality ,This Bluetooth smart watch offers a better touch experience and remarkable immersion and ,more than 200 watch faces for you to choose from App"Da Fit ", you can change the dial style according to your mood to wear keeping the world on your wrist
  • 【Accurate Activity Tracker Smart Watch 】sports watch activity tracker uses sweat and dust resistant breathable silicone strap, our Fitness Tracker Smart Watch for Android iOS Phones supports 12 sports modes,walking,running,bicycle,swimming,badminton,basketball,football,rope skipping. This bluetooth calls smart watch for men women is lightweight and stylish, bringing more convenient and relaxing sports experience while also providing more protection.
  • 【Your 24 Hour Personal Daily Health Monitor】Fitness watch for women men full touch screen IP67 waterproof bluetooth smartwatch for Android iOS Phones with heart rate blood pressure monitor multifunction Andriod smart watch with steps counter sleep monitor sports fitness watch, monitoring blood oxygen saturation allows you to understand your physical condition in real time to track fitness progress ,to know your health status in time.
  • 【More Functions Bluetooth Call Smart Watch】This fitness activity smart watch full screen for iPhone and Android phones includes a bunch of new features including:The bluetooth smart watch can hang up the phone, receive and display WhatsAPP /Text /Facebook /Twitter /Messenger reminder.Support sedentary reminder, alarm clock, stopwatch, timer,weather information,music control, remote camera control compatible with Android 5.0 and above or iOS 9.0 and above smartphones.

Streaming calls need a missing-data state

For streamed Chat Completions, request usage explicitly with stream_options: {"include_usage": true}. OpenAI documents that the usage object arrives in a final chunk before [DONE]. If the connection drops, that final chunk may never arrive. Mark the event as incomplete or unknown and queue it for reconciliation; never turn the absence of a final chunk into zero usage.

Keep partial text and transport telemetry separate from billing fields. A stream that delivered several visible tokens can still have an unknown billed total if the terminal usage record was lost.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Turn tokens into cost without false precision

Token-derived cost is an estimate unless it uses the exact model, token categories, and price rules that applied to the call. OpenAI’s API pricing documentation separates input, cached input, and output for many models and lists separate charges for some tools. Rates, tiers, and model availability can change, so store a price-table version and effective date with every calculation.

Use a versioned price table

Key each rate by at least:

  • provider and exact model ID;
  • token category, such as input, cached input, output, or reasoning output;
  • effective start and end dates;
  • context or usage tier;
  • service tier or endpoint terms where applicable; and
  • tool or modality charges that are billed outside token counts.

A calculation is conceptually:

derived_cost = (input_tokens × input_rate)
             + (cached_input_tokens × cached_input_rate)
             + (output_tokens × output_rate)
             + reasoning_or_other_token_charges
             + tool_or_modality_charges

The rates in that formula must be looked up by the exact model and price version. Do not publish a universal “cost per 1,000 tokens” figure without naming the model, category, tier, endpoint terms, and date checked.

Prefer provider-reported amounts when available

If the provider returns a charge or an authoritative usage breakdown, retain it and use it for reconciliation. Keep your derived amount too: it exposes price-table mismatches and lets you report an estimate immediately when an invoice-level amount arrives later. For custom, private, or self-hosted models, maintain a custom definition or ingest the owner’s cost because a public price table may not exist.

Rank #3
New Bee Guitar Cable 10ft Electric Instrument Bass AMP Cord | 6.35mm Jack, Durable Design, Stage and Home Studio Use, Low Noise Shielded Electric Cord for Bass, Keyboard, Amp, Mixer
  • [Low Noise High Fidelity] New bee guitar cord with 22AWG oxygen-free-copper center conductors provides enhanced signal transfer, the PVC insulation material and the high-density stranded copper double shield provide noise rejection for quiet operation
  • [Ideal for Choice] New bee 1/4 straight and right angle metal connectors, ensures a secure connection in any instrument, pedal, or amp jack. L shape great fro on stage guitar stands
  • [Wide Compatible] Suitable for mixer, power amplifier, microphone, effector, decoder, equalizer, electoronic organ, electric guitar, electric bass and other electronic equipments. Widely used in the installation of smart home, karaoke speaker, professional stage audio, etc
  • [Easy Disassemble] Making incidental repairs quick and easy! Very nice, flexible cable
  • [Dont Hesite to Order] 24 hours customer services and professional technology team are standing by

Reasoning models require provider usage

Visible output is not a safe proxy for billed output on reasoning models. Langfuse documents that it cannot infer the correct cost for models such as OpenAI o1 when token usage is absent because hidden reasoning tokens also contribute to billed output. Ingest the provider’s token counts for these calls; if they are unavailable, report the amount as unknown or an explicitly bounded estimate rather than presenting a precise number.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Build the event table and rollups

An append-only event table is easier to audit than a table that stores only daily totals. One possible relational shape is:

Column group Example columns Purpose
Identity event_id, parent_request_id, trace_id Connect retries and secondary calls to one product request
Attribution user_id, feature_key, tenant_id, environment Answer product, customer, and deployment questions
Provider provider, model_id, endpoint, service_tier Match usage and prices correctly
Usage input_tokens, output_tokens, cached_input_tokens, reasoning_tokens, raw_usage_json Preserve all billed categories and future fields
Money provider_cost, derived_cost, price_version, currency Separate authoritative and calculated values
Quality usage_status, error_code, started_at, completed_at Find missing data, failures, and latency patterns

From this table, create rollups partitioned by date and grouped by user_id, feature_key, model_id, provider, and environment. Keep counts of complete and incomplete events alongside token and cost sums. A total that excludes unknown events should be labeled as a lower bound, not as final spend.

Useful views

  • User view: daily cost, token mix, request count, retry count, and top features.
  • Feature view: cost per request, median and high-percentile tokens, model mix, and failure rate.
  • Model view: input/output/cached/reasoning shares and estimated versus provider-reported cost.
  • Data-quality view: unknown-usage rate, unknown-model count, missing user or feature attribution, and rollup lag.

Alert on spend thresholds and on data-quality failures. A sudden increase in unknown usage can hide a billing problem even when the visible cost chart looks stable.

OpenAI dashboard: useful, but not a product ledger

OpenAI’s Usage Dashboard is a quick audit tool. Authorized organization owners or people granted Usage Dashboard permission can view data in UTC, select projects, and filter Responses and Chat Completions usage by user. The dashboard does not combine organizations, and OpenAI notes that some Scale Tier bundle costs are attributed to the organization rather than to individual projects. For combined or custom reporting, OpenAI points to the Usage API. Details and current scope are in the official dashboard guidance.

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Rank #4
BSIMB 15.6" Digital Family Calendar & Chore Chart, No Subscription Fees
  • Digital Family Calendar: BSIMB digital calendar wall touch screen puts an end to your chaotic life. Designed for busy families, the digital calendar keeps you organised and in step with your family. Assign different colours and tasks to each family member to make family scheduling easier and more intuitive.
  • Easy Setup and Auto-Sync: Plug in, connect to WLAN and auto-syn your electric calendar in minutes. Digital wall calendar compatible with Google calendar, iCloud calendar, Outlook calendar, Cozi calendar and Yahoo calendar. Add events directly on the device or through the ecalendar app. Electronic calendar stays current whether you're at work or traveling—no manual updates needed.
  • All-In-One Home Assistant: Smart calendar builds habits and encourage independence with personalised chores. Display daily meal plan to reduce the daily ‘What's for dinner?’ stress. To-do lists keep track of everything you want. Turning chores into valuable achievements with stars and rewards, making life easier and more enjoyable.
  • Manage Anytime Anywhere: BSIMB electronic calendar for family stay connected with the powerful ecalendar app. Access and manage your digital wall calendar anytime, anywhere with free features like stars & rewards, photo screensaver, chore charts, meals plans, to-do lists etc. Electronic planner paid features like Magic Import tailor advanced tools for users with special needs.
  • Stay in Touch with Loved Ones: Share photos or videos from your phone to electronic wall calendar with app, instantly update your life moment, and keep in touch with your family, especially those who are far away; you can also surprise your smart calendar digital wall by sending him a wish card; it's also great for keeping tech-illiterate elders in touch with the youngsters.

Use the dashboard to compare your application ledger with provider-side totals, investigate a date range, or check whether a project is receiving traffic. It cannot replace application instrumentation when you need a feature key, your own tenant hierarchy, cross-provider joins, or a consistent treatment of retries and background calls.

Three implementation approaches

Approach Best fit Strengths Limitations
Provider dashboard and response fields One provider, quick audit, or an early implementation Low setup; provider usage fields and user filters are available for supported OpenAI endpoints Project and organization boundaries may not match product features; cross-provider and custom dimensions require your own aggregation
Application-owned events and rollups Custom feature billing, cross-provider reporting, or strict retention requirements Full control of attribution, retention, raw evidence, pricing logic, alerts, and exports You must instrument every call, maintain prices, handle missing streams, and operate dashboards and reconciliation
LLM observability platform Teams that want traces and ready-made breakdowns Trace context, user and tag dimensions, dashboards, alerts, and aggregate APIs can reduce dashboard work Validate integration coverage, data retention, throughput, privacy controls, pricing tiers, and whether exact provider usage is ingested for every model

Choose based on the dimensions and accuracy you require, not on the presence of a token chart. A platform that only infers tokens from text may be unsuitable for reasoning models or any provider with hidden billing categories.

Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

Using Langfuse for user and feature breakdowns

Langfuse’s documentation describes three required elements on a generation observation for cost breakdowns: model name, usage details, and optional explicit cost. Business dimensions such as users and features can be represented with tags set when the observation is created. See its trace best-practices documentation for the generation attributes and its token and cost tracking documentation for ingestion and aggregation behavior.

  1. Create a generation observation around each provider invocation.
  2. Set the exact model name and attach a stable user identifier and feature tag at creation time.
  3. Send normalized input, output, and cached-token usage where the provider returns it; preserve reasoning and other categories instead of folding them into a generic total.
  4. Ingest the provider’s cost when available. Otherwise, associate a model definition with the correct price rules and effective date.
  5. Use the Metrics API to aggregate by user or tags, then compare those aggregates with your own event ledger or provider dashboard.

Langfuse states that ingested usage and cost take priority when both are supplied. For private prices, custom models, or self-hosted endpoints, add a custom model definition or ingest the amount directly. Its documentation also says inferred cost for a reasoning model is not reliable when the provider does not supply usage because hidden reasoning tokens are not visible in the text.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Commercial terms are volatile. The Langfuse pricing page viewed on October 13, 2026 displayed plan-dependent included usage, history windows, ingestion limits, and additional-usage pricing, including 50,000 units for Hobby, 100,000 included units for paid plans, and $8 per additional 100,000 units on Core, Pro, and Enterprise at that time. Verify the live pricing page before budgeting; those figures can change.

Reconcile application data with provider data

Reconciliation catches lost stream terminators, wrong model IDs, duplicate retries, and price-table errors. Run it on a defined UTC window and keep the comparison dimensions identical.

  1. Export provider usage for the same projects, endpoints, models, and dates.
  2. Aggregate your event table using provider-compatible categories and UTC boundaries.
  3. Compare request counts, input tokens, output tokens, cached tokens, and cost separately. A matching total can conceal a category mismatch.
  4. Investigate unknown events, events created without a user or feature, and model IDs that have no price-table row.
  5. Record an adjustment or reconciliation reference rather than rewriting the original event.

Do not expect every provider total to map to a customer. Organization-level charges, prepaid bundles, taxes, credits, and other billing categories can sit outside per-request token usage. Label those amounts separately so a product feature report is not mistaken for an invoice.

Privacy, security, and retention decisions

  • Use opaque internal user IDs or tenant IDs; avoid email addresses in tags unless there is a documented need and access control.
  • Keep prompt and completion content out of cost dimensions. Store content only when required for debugging, with redaction and retention rules.
  • Restrict cost dashboards by tenant and role. A support agent should not automatically see another customer’s usage.
  • Encrypt raw usage responses and define a deletion or retention policy that matches contractual and regulatory requirements.
  • Log who changed a price-table row. A rate edit should not silently rewrite historical amounts.

Troubleshooting common monitoring failures

Every feature shows zero cost

Check that the wrapper receives the feature key before the provider call, that asynchronous jobs propagate context, and that rollups include the environment where traffic runs. A missing dimension should be reported as an attribution error, not grouped into an apparently free feature.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Streaming totals are lower than the dashboard

Look for interrupted streams whose final usage chunk never arrived. Mark them unknown, use provider-side data for reconciliation, and monitor the incomplete-usage rate. Never backfill zero simply to make a chart balance.

Calculated cost disagrees with the provider

Verify the exact model ID, input versus cached-input category, output or reasoning category, service tier, tool charges, and effective price date. Keep the raw response and price version so the discrepancy can be reproduced.

A reasoning model appears unusually cheap

Confirm that reasoning-token usage was returned and stored. If only visible output was captured, the amount is incomplete; obtain provider usage or mark the cost unknown.

Cross-provider totals cannot be compared

Normalize event shape and currency, but do not force unlike categories into one token column. Preserve provider-specific fields and report a common subtotal only when its definition is explicit.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

A practical rollout plan

  1. Start with one service. Instrument its wrapper and verify that every invocation creates an event, including errors and retries.
  2. Add attribution tests. In automated tests, assert that known users and feature keys survive synchronous, asynchronous, and streaming paths.
  3. Store raw and normalized usage. Add an unknown state before enabling cost alerts so missing data cannot be mistaken for zero.
  4. Version pricing. Load model and category rates with effective dates, then test a known response against the expected calculation.
  5. Publish two dashboards. One shows product dimensions; the other shows data quality and reconciliation status.
  6. Expand provider coverage. Add new models only after their usage categories, tool charges, and private-price rules have a defined mapping.

Once this foundation is in place, per-user and per-feature reporting becomes a query over trustworthy events rather than a guess based on visible text or a provider-wide total.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

AI cost monitoringLLMobservabilityOpenAI APIToken usageusage attribution

Leave a Reply

Your email address will not be published. Required fields are marked *

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Recommended PC Tool
Recommended PC Tool
Outdated Drivers Are Slowing You DownFree scan - exact matches
Windows Errors? Fix Them Before They SpreadFree repair scan

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.