Free tools Windows power users keep installed
One-click scans. No signup required.
When an AI agent behaves unexpectedly in production, an end-to-end trace can help you find where the run went wrong. Instrument model calls, tool use, handoffs, policy checks, and important downstream operations; connect those spans to structured logs and operational metrics. Add quality and security evaluation, and decide how to protect prompt and response content before enabling its capture.
What should you monitor in an AI agent run?
An agent run is a workflow, not just a model response. A trace should represent the user request or background job and show the steps that contributed to its outcome. OpenAI’s Agents SDK documents traces containing model generations, tool calls, handoffs, guardrails, and custom operations. Microsoft’s security guidance likewise recommends linking steps in an end-to-end request trace. OpenAI Agents SDK tracing · Microsoft guidance on AI system observability
Represent the workflow, not only the final answer
Start a trace at the boundary where a request or job enters the system. Add child spans for each consequential operation, such as retrieval, model generation, a tool invocation, a handoff to another agent, and a guardrail or policy decision. Include useful context such as service and agent identity, timestamps, run or conversation identifier, framework and model version where appropriate, and the tool’s identity and permission context.
Microsoft recommends capturing request identity context, timestamps, run or conversation identifiers, retrieval provenance, and tool names, arguments, permissions, and outputs, subject to governance controls. These details make it possible to reconstruct a path through the system instead of guessing from the final response. Microsoft’s observability guidance
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Clear out junk files and repair common Windows errorsFree Scan →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →#1 Best Overall
Use OpenTelemetry as a foundation, not a guarantee of identical agent fields
OpenTelemetry supplies foundations for traces, metrics, and logs, and its agent-observability overview describes both built-in instrumentation and instrumentation-library approaches. However, conventions for agent frameworks are still evolving; do not assume that different frameworks emit the same span names or attributes. Check the current conventions, framework version, and exporter configuration for the stack you deploy. OpenTelemetry’s 2025 overview of AI agent observability
How do you connect logs to traces across services?
Propagate trace context across service boundaries and include TraceId and SpanId in log records where supported. Add resource context identifying the emitting service or deployment. OpenTelemetry identifies trace context, execution time, and resource context as useful correlation dimensions; with them, an operator can move from a log entry to the relevant span and see which component emitted it. OpenTelemetry logging specification
Rank #2
Check remote tools and MCP boundaries
A trace can become incomplete when an agent calls a remote tool or MCP server. Verify that the caller propagates context and that the receiving service records compatible spans. Microsoft Agent Framework documents propagation of OpenTelemetry trace context to MCP servers when an active span context exists. Confirm that behavior in the versions and configuration you actually deploy rather than assuming propagation across every integration. Microsoft Agent Framework observability documentation
Include enough context to explain the run
Use identifiers to correlate events, but avoid putting secrets or full content into every log record. Useful structured fields can include operation name, outcome, duration, service or agent identity, tool identity, and a run identifier. The exact field names depend on the instrumentation; the key is to make the same execution navigable across logs, spans, and services.
Should production logs include prompts, responses, and tool content?
Not by default. Prompts, model responses, and tool arguments or results may contain personal information, credentials, confidential business data, or material that could expose an application to further risk. First decide what you need to diagnose incidents, then define redaction or sampling, access controls, storage location, retention, and deletion rules. Microsoft advises governing collection and retention through data contracts that balance forensic needs with privacy, residency, minimization, retention requirements, and legal obligations. Microsoft’s organizational guidance
Inspect framework defaults before enabling content capture
Defaults differ by SDK, version, and backend. Microsoft Agent Framework documents ENABLE_SENSITIVE_DATA as false by default and warns that enabling sensitive content can expose secrets; its guidance is to use that setting only in development or test. The OpenAI Agents SDK for Python documents trace_include_sensitive_data as true by default; disabling it omits Responses API request input and response output from those spans. Treat these as framework-specific settings, verify the exact version and configuration in use, and confirm what the destination backend retains. Microsoft Agent Framework observability documentation · OpenAI Agents SDK tracing documentation
Keep necessary content out of general-purpose log entries
When content is required for debugging or incident response, consider whether it belongs in a separately controlled store rather than ordinary logs. Google Cloud recommends storing prompts and responses in Cloud Storage instead of log entries, noting that its Cloud Logging maximum log-entry size is 256 KiB. That limit and storage recommendation apply to Google Cloud’s services; they are not universal requirements for other platforms. Google Cloud’s AI agent observability documentation
Which metrics and evaluations should you add?
Tracing explains the sequence of events; metrics show whether the system is operating within expected bounds. Build dashboards around the service’s operational objectives and establish a baseline for normal behavior before setting alerts.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Scan for outdated or missing drivers - takes under a minute3Clear out junk files and repair common Windows errors- Latency: Track end-to-end run duration and, where useful, time spent in model calls, tools, retrieval, or other major operations.
- Errors: Measure failed runs and failures by operation or service so a tool outage is distinguishable from a model or orchestration problem.
- Volume: Track request and tool-call volume to spot changes in demand or behavior.
- Token use and cost signals: Monitor these where the framework or provider exposes them, and associate them with the relevant service or workload.
- Quality and safety: Evaluate outcomes such as groundedness, safety or risk, and correctness of tool use. Use repeatable regression runs or release gates where appropriate.
- Security events: Monitor relevant abuse scenarios, including prompt injection and data exfiltration, with enough governed context to investigate them.
Logs and traces alone do not establish that an answer was accurate or safe. Pair runtime telemetry with repeatable evaluation and review of policy decisions; alert on meaningful deviations from baselines and service objectives rather than every unusual agent action. Microsoft’s AI observability guidance
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.How should you choose a telemetry backend?
Choose based on the frameworks, languages, and runtimes you operate, and on the controls you need for data handling. OpenTelemetry-compatible export can help preserve portability, but it does not ensure that every framework and service produces a complete trace or that all backends offer the same controls.
- Instrumentation coverage: Does the integration cover your framework, language, runtime, model calls, tools, and workflow operations?
- Trace completeness: Can you see failures, handoffs, remote calls, and downstream service activity? Does context survive service boundaries?
- Portability: What OpenTelemetry instrumentation and export paths are available, and which fields or features are specific to the provider?
- Content controls: Can you configure prompt and response capture, redaction, sampling, and access?
- Data governance: Check retention, deletion, residency, encryption, and access controls against your requirements.
- Operations: Consider integration effort, evaluation and alerting support, and the cost of operating the chosen setup.
Provider documentation illustrates different integration paths, not an independent ranking or apples-to-apples performance comparison:
| Provider documentation | Documented example | What to verify for your deployment |
|---|---|---|
| Amazon CloudWatch | OpenTelemetry traces from multiple agent frameworks and compute environments. | Framework and runtime coverage, trace completeness, exporter setup, and data controls. |
| Google Cloud | OpenTelemetry instrumentation for LangGraph and ADK, with trace analysis. | Whether the integrations cover the agent workflow and services you use, and how content and retention are controlled. |
| Microsoft Foundry | Native tracing integrations for Microsoft Agent Framework and Semantic Kernel, plus instrumentation paths for other frameworks. | Compatibility with your framework and exporter configuration, as well as the data-handling settings for the destination. |
These documentation examples do not establish neutral vendor rankings or a universal retention period, legal requirement, or cost comparison. Confirm the controls and capabilities for your own region, service, and configuration.
How do you validate an agent trace before relying on it?
- Generate a representative run. Exercise a normal path with a model call and tool use, plus a failure or handoff if those are part of the workflow.
- Inspect the trace in the selected backend. Confirm the expected model, tool, guardrail, and custom-operation spans appear in order, and that failures are visible.
- Follow a log-to-trace link. Check that trace and span identifiers and resource context let you locate the relevant execution across components.
- Test cross-service propagation. For remote tools or MCP servers, verify the receiving service records compatible spans and the run remains correlated.
- Review captured data. Confirm the content present in spans and logs matches your redaction, access, retention, and deletion policy.
- Check the quality signal path. Verify that evaluation results and operational alerts are visible to the people responsible for incident response and release decisions.
Microsoft Foundry documentation says traces typically appear in its portal within 2–5 minutes. That timing is specific to Foundry and may change; use the service’s current documentation when setting expectations for a different platform. Microsoft Foundry tracing documentation
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
