Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →Scan for outdated or missing drivers - takes under a minuteDriver Scan →To stop a long-running AI agent from forgetting earlier decisions without resending every turn, assemble a bounded prompt from a short-term working set, a compact summary, and only the durable memories relevant to the current task. Keep authoritative documents in permission-controlled knowledge sources and retrieve them when needed. Then measure whether the system still recalls the right information—not just how many tokens it saves.
Why does an agent’s prompt grow, and why does that matter?
A model’s context window is a bounded working set, not durable memory. System instructions, recent conversation, retrieved material, tool results, and the model’s response all compete for that space. Microsoft’s multi-agent reference architecture recommends budgeting for the response as well as the prompt; filling the window with input leaves less room for useful output.
As an Amazon Associate I earn from qualifying purchases.
If an orchestrator resends the full conversation at every turn, the input grows as the session gets older. AWS’s Well-Architected Agentic AI Lens identifies repeated history as a source of rising token use and eventual context pressure. The bill for a session cannot be inferred from transcript length alone: model pricing, caching, output length, tool use, and call frequency also affect cost.
There is a compounding effect: older material is charged or processed again even when it is irrelevant to the current step. The 2026 preprint Agentic Context Management analyzes how naive context accumulation can make token cost grow quadratically over a conversation, while crude summarization can reduce cost at an accuracy cost. That is the paper’s analysis, not a billing law that applies identically to every model or orchestration setup.
#1 Best Overall
What should an agent remember, and what should it retrieve?
“Memory” covers different jobs. Separating them makes it easier to decide what belongs in the prompt, what belongs in durable storage, and what should remain in a source of truth.
| Information type | Best role | Typical handling |
|---|---|---|
| Active session state | Support the current task and its immediate follow-up turns. | Keep relevant recent turns, current task status, and unresolved items in a bounded working set. |
| Durable semantic memory | Preserve compact, reusable facts or preferences across sessions. | Store only information intended for future recall, with provenance and access scope. |
| Episodic memory | Recall a prior interaction when continuity across sessions matters. | Retain or index selected interaction history; retrieve relevant episodes on demand rather than injecting the full archive. |
| Procedural knowledge | Explain how a documented process or tool works. | Keep it in documentation, a knowledge base, or an appropriate tool, and fetch the necessary part when needed. |
| Organizational records | Provide authoritative, permission-controlled facts that may change. | Keep the source of truth in its governed system and retrieve it at query time, when freshness and access checks can be applied. |
Microsoft’s memory guidance distinguishes agent memory from authoritative knowledge bases. Copying company documents into personal or conversational memory can create stale copies and bypass the controls of the original source. Store a compact pointer or task-relevant fact when appropriate; retrieve governed content from its source when the agent needs it.
How should a bounded context be assembled?
Treat each model call as a context-assembly operation. Set a total input budget below the model’s hard limit, reserve room for the expected response, and give every injected section a cap. Microsoft’s reference recommends a structured order and warns that uncapped profile or retrieval content can crowd out the active conversation.
Rank #2
- Reserve fixed room first. Account for system instructions, tool definitions or other required setup, and output headroom before allocating tokens to memory. The exact allocation depends on the model and task.
- Add a capped stable profile only when useful. Include a small set of durable preferences or facts relevant to this agent’s work. Do not let profile text expand without a limit.
- Retrieve selectively. Search durable memory or an authoritative knowledge source for the current task, apply relevance checks, and cap the number and size of retrieved items.
- Include a bounded recent-turn window. Keep recent exchanges verbatim when their exact wording, sequence, or detail matters to the task.
- Add a compact summary of older conversation. Preserve decisions, constraints, identifiers, and open items explicitly rather than relying on a vague narrative recap.
- Check the assembled prompt before sending. Record the size of each section and the total, then trim or retrieve less if the budget is exceeded. Keep source labels with each block so the model and operators can distinguish user statements, summaries, and retrieved material.
Microsoft gives 70–80% of the context window as an example threshold for triggering summarization before the hard limit. It is a starting point, not a universal optimum: output size, tool flow, and workload affect how much headroom a system needs.
Which strategy fits a given agent?
No strategy is best for every workload. Choose according to the task’s recall needs, the cost of missing a detail, and the operational constraints.
| Strategy | Useful when | Main cost or risk |
|---|---|---|
| Full-history context | Sessions are short, or the task genuinely requires the complete interaction. | Repeated input grows with history and can increase token use or eventually exceed context capacity. AWS discusses these risks in its agent memory cost guidance. |
| Sliding window | The latest turns dominate and earlier details are rarely needed. | Earlier decisions and constraints can disappear from the prompt even when the user expects continuity. Microsoft’s short-term memory guidance describes this context-management trade-off. |
| Rolling summary plus recent turns | A long active session needs continuity without carrying every turn verbatim. | A summary may omit details. Preserve critical state explicitly and retain source history for drill-down. Microsoft’s short-term memory guidance covers rolling summaries and incremental updates. |
| On-demand retrieval from durable memory | Cross-session recall is needed, or the archive is too large to include on every task. | Irrelevant or missing results, retrieval latency, and indexing or storage costs must be monitored. AWS’s cost and performance guidance recommends selective retrieval and observability. |
| Long-context model, optionally with caching | A task repeatedly uses large fixed inputs or requires reasoning over a large document set. | A larger window does not guarantee perfect multi-fact retrieval or eliminate per-request cost. Cache behavior and price are provider- and model-specific; Google documents caching as an optimization, not a universal saving. |
A long context window can reduce the need to aggressively prune some tasks, but it is not a substitute for memory design. Google’s Gemini API guidance notes that retrieval performance can vary when a model must find multiple pieces of information in a large context. Its page also gives a provider- and model-specific illustration in which cached use of Gemini Flash costs approximately four times less than the described uncached scenario. That comparison is conditional on the page’s caching scenario and should not be generalized to other models, workloads, or current prices.
How should summaries and durable memories be maintained?
Compact the active session before it reaches the limit
Summarize incrementally rather than waiting until the prompt is nearly full. Retain the recent turns needed to resolve local references, while the summary carries older state forward. Microsoft’s short-term memory guidance recommends preserving decisions, constraints, identifiers, and open items; those are more actionable than a general recap of what was discussed.
Promote information deliberately
Not every conversational detail belongs in long-term memory. Define which facts are temporary session state, which may be promoted for future use, and which should remain only in a source document or raw transcript. Promotion should have a reason: recurring preference, a durable project decision, or another fact that future tasks actually need.
Keep provenance, permissions, and deletion in the lifecycle
For each stored memory, retain enough provenance to know where it came from and what access scope applies. Retrieved user text and documents are data, not trusted instructions. Specify how transient state expires, what is archived, how a memory can be corrected or deleted, and whether raw transcripts are retained separately from extracted facts. These choices affect privacy, freshness, and the ability to honor access changes.
Rank #4
How can teams tell whether memory is working?
Measure useful recall per token and end-to-end performance, not compression ratio alone. AWS recommends per-agent and per-session cost visibility alongside retrieval relevance and performance monitoring. A prompt that is cheap because it discarded the key constraint is not an optimization.
- Recall quality: Can the agent recover decisions, constraints, identifiers, and prior facts needed for representative tasks?
- Tokens per successful task: Count input and output tokens across the full task, including retries and tool calls, rather than comparing only summary size.
- Latency: Record retrieval and total response time; more selective memory can still add a retrieval step.
- Retrieval noise: Inspect whether retrieved items help answer the current task. Monitor top-k count, chunk sizing, and relevance rather than assuming more results are better.
- Operational cost and control: Track storage and indexing needs, freshness, access enforcement, lifecycle behavior, and implementation complexity.
Test these measures on representative workloads, including cases where the necessary fact is old, where several facts must be combined, and where no stored memory should be used. Compare architectures on the same tasks. AWS’s Well-Architected Agentic AI Lens puts the principle succinctly: “Retrieval is selective rather than bulk, so the tokens spent per reasoning step reflect the context that actually matters to the current task.”
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
What do memory-compression results establish?
Research shows that large reductions in stored or retrieved context are possible, but results depend on the corpus, representation, and retrieval method.
The authors of the March 2026 preprint Structured Distillation for Personalized Agent Memory: 11x Token Reduction with Retrieval Preservation studied 4,182 conversations and 14,340 exchanges across six software engineering projects. They reported an average representation of 38 tokens per exchange instead of 371. In their best pure-distilled configuration, mean reciprocal rank (MRR) was 0.717 versus 0.745 for the best verbatim baseline—96% of that baseline. The paper reports different outcomes across retrieval mechanisms, including BM25 and vector search. These are results for that corpus and setup, not a general promise that an agent can compress any history elevenfold without meaningful recall loss.
Microsoft Research’s PlugMem work evaluated long multi-turn conversation questions, cross-article fact finding, and decisions while browsing. Microsoft Research reports that PlugMem outperformed generic retrieval and task-specific designs while using less memory context. Treat that as the authors’ result on their reported benchmarks, not an independent replication or a guarantee for another agent’s workload.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
