Recommended Free Tools
An agent that misses a refund window, selects the wrong SKU, forgets a tool result, or overlooks a customer exception may have a retrieval problem—but that is only one possibility. The useful first question is not “Which model should I switch to?” It is “What did the agent actually receive?”
The 49% figure in this story has a specific source and scope: Anthropic reported that its Contextual Retrieval method produced 49% fewer failed retrievals in its evaluation, rising to 67% fewer when reranking was added. Those are retrieval results, not a promise of fewer agent task failures. They prompted a more disciplined debugging approach: trace the context, locate the failure stage, and test retrieval changes against representative cases.
As an Amazon Associate I earn from qualifying purchases.
Why an apparently obvious task can fail
An agent cannot use a policy, identifier, exception, or earlier tool result that never reaches it in a usable form. That makes context acquisition a plausible upstream failure: the model may be capable of answering correctly, yet the relevant evidence was absent, filtered out, buried below the retrieval cutoff, or lost while context was assembled.
The Tool Desk
Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →But an incorrect answer does not by itself prove retrieval failed. The evidence might have been present and ignored; a tool could have been called incorrectly; the agent might have misunderstood the tool’s output, made a bad plan, or run into a policy constraint. Treat retrieval as a hypothesis to test, not a catch-all diagnosis.
What the 49% figure does—and does not—mean
In its September 19, 2024 article, Anthropic reported 49% fewer failed retrievals with Contextual Retrieval, and 67% fewer when reranking was added. Its method adds chunk-specific explanatory context before creating contextual embeddings and a contextual BM25 index; reranking is an additional step for ordering candidates.
The metric is failed retrievals in Anthropic’s evaluation. It is not the percentage of agent failures caused by retrieval, a universal accuracy gain, or a result guaranteed for another corpus. The first-person account that inspired this topic uses the figure to motivate a debugging method; it does not report a controlled test showing that the author’s own agent failures fell by 49%.
Start by preserving what the agent actually saw
For one failed run, save the full trace before changing prompts, models, or indexes. The database and intended prompt are not the same thing as the assembled context passed to the model.
- The exact user query and relevant prior conversation messages.
- Retrieved documents and chunks, their ranking scores, applied filters, and the top-k cutoff.
- Index version or freshness information, plus any memory injected into the prompt.
- Tool calls and their raw outputs, including errors or missing results.
- System instructions and the final assembled context, in order, as passed to the model.
This trace lets you ask concrete questions: “what did retrieval return?”, “where did the fact appear in context?”, “did exact-match search exist?”, “was reranking applied?”, and “did the agent actually see the right thing in usable form?”
Classify the failure before choosing a fix
Microsoft Research’s AgentRx framework, announced March 12, 2026, illustrates why diagnosis needs more than a final success-or-failure label. It normalizes agent trajectories, checks constraints derived from tool schemas and domain policies, and uses an LLM judge to identify a critical failure step. In a benchmark of 115 manually annotated failed trajectories across τ-bench, Flash, and Magentic-One, Microsoft reported improvements over prompting baselines of 23.6% in failure-localization accuracy and 22.9% in root-cause attribution. These are Microsoft’s reported experimental results, not a guarantee for a different system.
Its taxonomy suggests useful branches for an incident review:
- Evidence acquisition: Was the needed source absent from candidate results? Check ingestion, chunk boundaries, filters, query wording, lexical coverage, and index freshness.
- Ranking: Was the source retrieved but placed below the context cutoff? Investigate ranking and whether reranking changes the order.
- Context use: Was the right evidence present but ignored or contradicted? Inspect context assembly, position, and whether the answer is grounded in it.
- State handling: Was the missing information a same-run tool result, a user preference meant to persist across runs, or external knowledge? Session state, durable memory, and corpus retrieval are separate systems.
- Planning or execution: Did the agent fail to adhere to its plan, invent information, invoke a tool invalidly, misread tool output, or mismatch its plan with the user’s intent?
- Intent or policy: Was the user’s intent underspecified or unsupported, did a guardrail trigger, or did the system fail? A better retriever will not necessarily fix these cases.
AgentRx’s announcement notes, “Traditional success metrics (like ‘Did the task finish?’) don’t tell us enough.” The point for debugging is to find the critical step and the category of failure before selecting an intervention.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →Choose retrieval tests that match the symptom
Exact identifiers need more than semantic similarity
Semantic search can find conceptually related text while missing an exact SKU, policy name, order ID, or error code. Compare it with lexical search such as BM25, or a hybrid approach that combines lexical and semantic candidates. Anthropic describes BM25 as useful for exact terms and technical phrases; Redis’s retrieval-debugging guide also separates hybrid search and ranking as distinct interventions. Evaluate these on a representative set of real queries and failures rather than assuming they will help every corpus.
Good candidates can still be ranked out
If the source appears among the candidates but not in the final context, test ranking and reranking. Keep the cutoff and token budget fixed while comparing results, so a better ordering is not confused with simply passing more material to the model.
Small, stable corpora may not need retrieval
Anthropic suggests that a knowledge base under 200,000 tokens—about 500 pages in its example—may fit directly into a prompt. That is Anthropic’s heuristic, not a universal threshold. Including a small corpus directly can remove retrieval plumbing, but it does not guarantee that the model will attend to evidence in every position or use it correctly.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Use benchmarks without mistaking retrieval for task success
The July 2026 Agent Retrieval Bench paper by Bowen Qin and Yi Xie measures file-level context retrieval for coding agents. It covers 427 samples across 25 repositories, with four positive retrieval task types and a selective-retrieval component. The authors report that logged trajectories miss every gold file on 27–35% of samples. They also report different leaders by metric: Qwen3-Embedding-4B for weighted MRR, Qwen3-Embedding-8B for weighted Recall@20, and RepoMap for budgeted context yield at 8K tokens.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Scan for outdated or missing drivers - takes under a minute3Clear out junk files and repair common Windows errorsThose results show measurable context-acquisition gaps, not that retrieval alone determines whether an agent produces a successful patch. The fact that different approaches lead on different metrics is a reason to evaluate against the tasks and constraints that matter to your system, not declare one retrieval family the overall winner.
A practical investigation sequence
- Reproduce one failure. Keep the exact input and preserve the full trace, including final assembled context.
- Locate the evidence boundary. Check whether the needed fact existed in the source, entered candidate results, survived ranking and filters, and reached the model.
- Separate retrieval from state and tools. Identify whether the missing item was external knowledge, cross-run memory, or a result produced earlier in the same run.
- Identify the failure stage. Distinguish missing evidence from poor ranking, ignored evidence, planning errors, invalid tool calls, execution faults, intent problems, and guardrails.
- Change one relevant component. For exact strings, compare lexical or hybrid retrieval; for candidates buried below the cutoff, test reranking; for stale results, inspect indexing and freshness.
- Compare on a representative set. Track retrieval quality separately from the final task outcome, and include cases where the agent should not retrieve or act.
Redis’s guide describes other lookalike causes, including missing chunks, generation that ignores good evidence, stale or duplicate indexes, and latency or execution failures. Its technical recommendations are vendor guidance; apply them to the architecture you actually run rather than assuming a Redis-specific result transfers elsewhere.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
