Free tools Windows power users keep installed
One-click scans. No signup required.
A trace shows that a model call happened. It does not, by itself, show what the model received. To check whether your prompt assembly is visible, open the recorded input of the model-call run and compare it with the exact message list your application sends at the last point before the call. If your system instructions, conversation history, retrieved context and tool descriptions appear there with the content you expect, the assembly is traced. If something is missing, the cause is either the instrumentation or the assembly itself, and the steps below help you tell which.
What the recorded input has to contain
An agent request is usually assembled from several parts, and each one can be lost independently. Before you test, list the components your application adds and where each should appear in the recorded model-call input.
As an Amazon Associate I earn from qualifying purchases.
| Component | What to look for in the recorded input | Common reason it is missing |
|---|---|---|
| System instructions | The system or developer message text, including your placeholder marker | The instructions are built in a template that is never passed to the traced call |
| Conversation history | Earlier user and assistant turns, in their original order | History is truncated, summarised or stored separately before the call |
| Retrieved context | The text of the retrieved passages, not only the retrieval query | Retrieval runs in a step whose output is never handed to the model wrapper |
| Tool descriptions | Tool names and descriptions, either in the messages or in a separate tools parameter | Tools are bound on a client object that sits outside the traced call |
Tool descriptions are the component most often overlooked, because many frameworks send them as a separate request parameter rather than as a message. Check that parameter as well as the message list.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
How to run the check
- Find the model-call boundary. Locate the last point in your code where the complete message list, or request object, exists immediately before the model is called. Log or copy that payload. It is your reference, because the trace has to be compared against what actually leaves your application.
- Enable instrumentation on that call. LangSmith documents SDK-based trace logging for use outside LangChain. Its reference describes a
traceabledecorator and a wrapped model client as two approaches. Apply one of them to the function or client that makes the model call, not to a parent function that only builds the prompt. - Open the model-call run. In the LangSmith interface, open the individual run for the model call rather than the top-level trace summary. Look at the recorded inputs or prompt-related fields for that run. The trace tree tells you the call happened; the run’s input tells you what was recorded.
- Compare contents, not identifiers. Check that the recorded roles and message text include the assembled instructions and relevant context, and compare them with the reference payload from step 1. Matching trace IDs or span names do not count as a match.
- If you use OpenTelemetry, confirm the emitted attributes. LangSmith documents OpenTelemetry trace ingestion and a set of prompt-related attribute mappings, including one for prompt-template variables. Confirm that your instrumentation emits content attributes at all, and that the backend you use maps and displays them. Some instrumentation emits only metadata, and a trace with no content attributes shows nothing about the prompt.
- Run one controlled request first. Send a harmless test prompt that contains a recognisable marker in each component: for example,
SYS-MARK-01in the system instructions,HIST-MARK-02in a prior turn,CTX-MARK-03in a retrieved passage andTOOL-MARK-04in a tool description. Confirm that each marker appears in the recorded model-call input. This is a standard diagnostic approach, not a benchmark, and it is worth repeating after any change to prompt templates or framework versions.
Reading the result
The marker test gives four possible outcomes, and each points to a different fix.
#1 Best Overall
- Marker in your reference payload and in the trace. Assembly and capture both work for that component. Repeat the check for production traffic only if the component’s source or instrumentation changes.
- Marker in your reference payload but absent from the trace. This is a capture gap. Confirm that the decorator or wrapped client wraps the real model call, check that your OpenTelemetry exporter emits content attributes, and confirm that the backend displays them.
- Marker in the trace but absent from the reference payload. The instrumentation is recording an object from an earlier layer, and something between that layer and the model call changes the message list. Move the instrumentation closer to the request, or add a log at the boundary to find the step that drops or rewrites the content.
- Marker absent from both. The problem is in assembly, not in tracing. The component is never added to the request, or it is added to a variable that is later overwritten. The trace is accurate; the prompt is incomplete.
Choosing how to capture prompts
The main decision is which route to use. The sources support two routes for LangSmith-based setups, and each leaves some questions open that you need to verify in your own environment.
| Route | Framework coverage | Model-call payload captured | Backend displays contents | Control over pipeline | Privacy and retention controls |
|---|---|---|---|---|---|
SDK tracing (traceable decorator or wrapped model client) |
Documented for use outside LangChain; coverage for each framework not stated | Recorded inputs are documented as viewable on the run; completeness must be checked with a marker test | Displayed in the LangSmith run view | Limited to what the SDK exposes | Not stated in the sources reviewed; verify in your account settings |
| OpenTelemetry export | Depends on the instrumentation library you use; not stated for all frameworks | Depends on whether content attributes are emitted; not stated universally | Depends on the backend’s attribute mapping | High, through your own collector configuration | Depends on your collector and backend; not stated in the sources reviewed |
LangChain’s LangSmith product page describes the goal as “See exactly what your agent is doing step by step.” That is vendor positioning about inspecting agent activity. It is not a guarantee that every prompt is captured in every configuration, which is why the marker test in step 6 matters more than the product description.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Handling captured prompt content
Once prompts are captured, they may contain user data, internal instructions or retrieved documents, so treat the trace store as sensitive. The sources reviewed do not establish the retention period, redaction options or data residency for any particular deployment. Check those settings in your own account or self-hosted configuration before sending production prompts to a tracing backend. Keep the marker strings in your test prompts free of real identifiers and secrets, and use the same care with any sample traces you share.
Recommended Free Tools
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
