Useful LLM observability needs more than a model-call log: trace the full workflow, use provider-reported billed token counts when available, choose sampling based on what you can afford to miss, and keep prompt and response content out of telemetry by default. OpenTelemetry’s GenAI semantic conventions offer a shared vocabulary for this work; MLflow and Amazon OpenSearch Service document two ways to apply it.
What should an LLM trace capture?
Trace the work around a model call, not just the call itself. An agent workflow may include orchestration, model requests, tool invocations, and retrieval. A hierarchical trace makes it possible to see which steps contributed to latency, token use, or an unexpected outcome. Amazon OpenSearch Service’s AI observability documentation describes traces for these kinds of agent steps, alongside OpenTelemetry integration and querying with PPL.
As an Amazon Associate I earn from qualifying purchases.
For cost analysis, capture enough non-content context to explain what happened:
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
- The operation or workflow step.
- The provider and the exact requested model name, when supplied by the vendor.
- Provider-reported token usage, with the usage type and whether it is billed usage.
- Related orchestration, tool, and retrieval spans that put the model call in context.
Use OpenTelemetry GenAI semantic conventions as the cross-platform vocabulary, but check the status and exact field names of the convention version your instrumentation implements. The conventions are maintained on a changing branch, so a field name or status found there may not match a released version supported by a given library or backend.
#1 Best Overall
- Dell Precision 7920 Tower Workstation
- 2x Intel Xeon Gold 6130 16-Core 2.1GHz (3.7GHz Turbo)
- 192GB DDR4 Memory - upgradable to 1.5TB
- 2x 1TB SSD + 2x 4TB HDD (Removable Hot Swap Drive bays)
- Nvidia Quadro P1000 4GB - Windows 11 Professional 64-bit
How do I track LLM token usage and cost in traces?
Prefer billed usage when providers report it
OpenTelemetry’s GenAI conventions say input-token totals should include all input types, including cached tokens. If a provider reports both billed usage and the number of tokens consumed by the model, use billed usage for charge-aligned accounting. The two counts can differ; record which one your telemetry represents instead of treating “tokens” as an unambiguous measure.
Keep totals and breakdowns distinct. Detailed usage attributes are subsets of totals, so adding a cached-token or other token-type breakdown to an already complete total would double-count usage. Explain what the total includes and present any breakdown as part of that total.
Separate measured usage from estimated cost
A token count reported by a provider is not the same thing as a cost calculated by your observability platform. Cost may be derived from a model price table, so label it as an estimate and retain the pricing assumptions used. Confirm that the price table and provider-specific usage behavior match your account and workload. There is no meaningful universal token price or cost figure without a model, provider, date, and pricing basis.
Free tools Windows power users keep installed
One-click scans. No signup required.
What MLflow documents
MLflow documents input, output, and total token counts for LLM calls, along with estimated cost in USD calculated from model pricing. The documentation describes viewing cost at both span and trace level. Its version requirements are specific: token tracking requires MLflow 3.2.0 or later; cost tracking requires 3.10.0 or later and the server’s [genai] extra. These requirements can change as the product evolves.
For Databricks managed MLflow, the documentation says cost computation requires LiteLLM or manually set cost attributes. It does not state that same requirement for self-hosted MLflow.
Rank #2
- [Local AI Inference & 70B Model Ready] Equipped with the AMD Ryzen 7 PRO 8845HS processor, NEXUS is engineered for heavy local AI workloads. With a full-size GPU bay, it runs 70B LLMs natively without an internet connection. Ideal for AI developers and tech enthusiasts who need private environment for coding and model testing.
- [132TB Mass Storage with ZFS Integrity] Features a hybrid storage architecture (3×NVMe + 4×3.5" HDD) supporting up to 132TB. Utilizing the enterprise-grade ZFS file system and ECC memory, it prevents data corruption and bit rot—a must-have for professional photographers and video editors safeguarding 4K/8K RAW footage.
- [OpenClaw-Driven Automation Workflow] The built-in OpenClaw execution layer allows complex automated tasks to be processed locally. Even when offline, your backup schedules and AI file organization continue seamlessly. Say goodbye to monthly cloud subscriptions and high latency.
- [Dual 10GbE & USB4 Ultra-Connectivity] Experience server-class speeds with dual 10GbE ports and a 40Gbps USB4 interface. It enables multi-user real-time collaboration on large project files directly from the NAS, ensuring zero-lag editing for creative studios and production teams.
- [Open-Source ZimaOS for Total Privacy] Running on the fully open-source ZimaOS, NEXUS ensures your data stays physically on-premise with no backdoors. It acts as a "Digital Fortress" for privacy-conscious families and small businesses who demand absolute data sovereignty.
Should I use head sampling or tail sampling for LLM traces?
Head and tail sampling make the keep-or-drop decision at different times. The right choice depends on whether you need a low-overhead representative sample or want to retain traces based on what happened later in the workflow.
| Approach | Decision timing | What it can preserve | Main trade-off |
|---|---|---|---|
| Head sampling | Early, often using trace ID and a configured probability | A probability-based sample selected before the full trace is available | Simple and efficient, but it cannot select based on downstream errors, full latency, or later span attributes. |
| Tail sampling | After all or most spans are available | Whole traces selected using outcomes such as errors, latency, attributes, or service-specific rules | More control, but requires stateful components, monitoring, and potentially significant resources at high traffic. |
| Combined sampling | An early decision followed by later trace-level selection | Traces that pass the early stage and meet later selection rules | Can protect a high-volume pipeline, but an early drop cannot be recovered by tail logic. |
When sampling is worth considering
OpenTelemetry’s sampling guidance identifies sampling as useful when most traffic is healthy and repetitive. It is less suitable when volume is already low, aggregate data can be pre-aggregated, or regulation prevents dropping data without a low-cost retention path. The guidance names 1,000 or more traces per second as one reason to consider sampling; this is an operational criterion, not a universal threshold.
The same documentation says high-volume systems may find that a sample rate of 1% or lower represents traffic. Treat that as implementation guidance, not a benchmark or a default target for every application. Sampling trades reduced observability volume against compute to sample, engineering effort to maintain policy, and the opportunity cost of missing useful information.
Choose rules that match the question
With head sampling, the decision happens before later spans reveal whether a trace failed or ran slowly. Tail sampling can preserve whole traces based on those later facts, but only if the required spans reach the sampler and it has sufficient state and capacity. In a combined setup, choose the early rate with the understanding that traces dropped there are unavailable to later rules.
OpenTelemetry identifies operation name, provider name, requested model, server address, and server port as attributes that can matter for sampling and should be available when the span is created if provided. A policy that samples particular model/provider groups or uses outcome attributes is an implementation choice; verify that the instrumentation emits the needed attributes before relying on them.
Rank #3
- Professional AI & Creator Workstation: AMD Radeon AI PRO R9700 GPU with 32GB GDDR6 is engineered for AI development, professional content creation, and compute-intensive workloads.
- Massive 32GB Memory Capacity: 32GB of GDDR6 memory on a 256-bit bus provides ample bandwidth for large AI models, 8K video editing, and complex 3D rendering.
- Advanced RDNA 4 with AI Accelerators: 64 Compute Units with 3rd Gen Ray Tracing and dedicated 2nd Gen AI Accelerators for groundbreaking AI performance and visual computing.
- Professional Blower Cooling: Efficient single blower design exhausts heat directly out of the chassis, ideal for multi-GPU workstation and server configurations.
- Enterprise-Grade Thermal Solution: Vapor chamber heatsink with industrial Honeywell PTM7950 thermal interface material ensures reliable cooling under sustained professional loads.
How do I keep prompts and responses private in observability traces?
Treat model instructions, user messages, and model outputs as sensitive by default. OpenTelemetry’s GenAI semantic conventions state: “OpenTelemetry instrumentations SHOULD NOT capture them by default, but SHOULD provide an option for users to opt in.” Tool outputs and retrieved context can contain sensitive material too, so a policy limited to prompt and response fields may still expose private data.
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →Minimize what enters telemetry
- Start with content capture disabled; enable it only for a deliberate, controlled purpose.
- Where feasible, redact or mask sensitive values before spans leave the application.
- Review tool results and retrieval context as well as model inputs and outputs.
- Set access controls and retention rules for telemetry, and limit access to any captured content.
Content can be large enough to exceed telemetry envelope or attribute limits. For production cases where volume or sensitivity matters, OpenTelemetry describes storing content externally and putting references on spans. Separate access controls can then protect the content independently from the trace metadata.
Masking is one safeguard, not a guarantee
MLflow publishes guidance for masking sensitive data from traces. Masking can be one layer in a data-handling design, but it does not prove that all sensitive information has been removed or that a deployment meets a legal requirement. Decide what must be excluded or transformed, where that happens, who can access the result, and how long it is retained.
How should teams apply these practices?
- Instrument the workflow. Create trace relationships for orchestration, model calls, tools, and retrieval so the trace reflects the whole unit of work.
- Record usage with its meaning. Capture provider-reported usage and distinguish billed counts from model-consumed counts when both exist. Make clear which token types the totals include.
- Keep derived costs identifiable. Label platform-calculated cost as an estimate and document the pricing assumptions behind it.
- Set a content policy before rollout. Disable prompt, response, tool, and retrieval content capture by default; define opt-in conditions and protections for any exception.
- Select sampling based on the information you need. Use early probability-based sampling where simplicity and pipeline efficiency matter; use tail rules when errors, latency, or later attributes must drive retention. Account for the traces an early filter can discard.
- Verify the implementation against supported versions. Check the semantic-convention version and exact attributes, as well as the current feature requirements of your tracing stack.
What the documented product examples establish
MLflow’s documentation establishes token tracking, estimated cost, and span- and trace-level views, with the version and deployment qualifications described above. Amazon OpenSearch Service documents hierarchical AI traces for agent workflows, model calls, tool invocations, and retrieval, along with OpenTelemetry integration and PPL querying. These examples show documented capabilities; they do not establish a comparative ranking of observability products.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
The Tool Desk
Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →
