Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →An LLM call is a computation over the text you send it. When that call ends, the model does not carry what happened into the next one. If an assistant seems to remember last week’s conversation, your application stored something, selected what matters, and placed it back into the prompt. Continuity is therefore a system design choice, and the quality of that design decides whether it works.
What the model sees on each call
A model’s working memory for a request is its context: the instructions, the messages, and any retrieved material included in that one request. Once the response is generated, the model does not update itself with the exchange. Send the same question to the same model with different context and you get a different answer, because the context is the only thing that changed.
As an Amazon Associate I earn from qualifying purchases.
This is why “memory” in a chat product is almost always application-managed state. The application decides what to keep, where to keep it, which pieces to fetch for the next call, and how to present them. The model then reasons over whatever arrived in that prompt.
The memory lifecycle
Treat memory as a repeating loop rather than a single feature. A workable implementation usually performs these six steps:
#1 Best Overall
- Decide what to retain. Choose which items persist: selected conversation turns, facts the user stated, task outcomes, or changing agent state. Most raw output should not be kept.
- Persist and index. Write each item to a store with a user, session, or task identifier and a timestamp. Depending on the access pattern, that store may be a transcript archive, a key-value table, a relational table, or a vector index.
- Retrieve. Before the next call, query recent state and longer-term memory for items relevant to the current input.
- Read and interpret. Turn retrieved records into usable text. Resolve conflicts, prefer the latest value of a changed fact, and drop items that no longer apply.
- Inject. Assemble the request from system instructions, a clearly labelled memory block, recent turns, and the current input.
- Generate and update. After the response, extract new facts, update structured state, write summaries where used, and record the outcome for future tasks.
AWS Prescriptive Guidance describes the same pattern for memory-augmented agents: retrieve recent and long-term state, place the memory context in the prompt, generate output, and store new information for later tasks. Its summary of the mechanism is direct: “The memory context is embedded into the LLM prompt, allowing the agent to reason based on both current inputs and prior knowledge.”
Step four is where many systems quietly fail. A store that keeps every old value will happily return both “the deadline is Friday” and “the deadline moved to Monday,” and the model has no reliable way to know which one is current unless the application resolved that before injection.
Choosing a memory design
The main options are not mutually exclusive, and many production systems combine them. Compare them on the requirements that matter for your product: retrieval quality, handling of changed information, latency, token and storage cost, auditability, user control, and the risk of mixing unrelated people or domains.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Repair Windows errors before they cause bigger problems3Fix the driver behind crashes, sound loss and screen glitchesAuto-injected curated layers
Here the application adds metadata, explicitly saved facts, recent summaries, and the current conversation to every request. Microsoft’s Memory Architecture Patterns guidance says this approach can make continuity feel seamless. Its costs are equally direct: the memory tokens are paid on every call, users have less control over what is included, and unrelated contexts can blend together.
On-demand retrieval
Here the application searches stored history or structured memory only when a request needs it. This avoids injecting the full past on every turn. The trade-off is dependence on indexing and ranking: if the right evidence is not surfaced, the model will answer without it. The LongMemEval paper frames long-term memory as three stages, indexing, retrieval, and reading, and it evaluates failures at each stage rather than only checking whether a string was stored.
Structured or extracted memory
Here the application stores selected facts, relationships, task outcomes, or changing state in a form it can inspect and update. Structure makes corrections and expirations possible in a way an undifferentiated transcript does not. It also adds work: someone, or some extraction step, has to decide what counts as a fact and how conflicting updates are resolved.
Full-context replay and summaries
Replaying full history gives the model the most raw material, but it consumes context quickly and is often impractical for long relationships. Summaries compress history into fewer tokens and cost less per call, but they discard detail, and a summary can introduce statements the original conversation never contained. Use full replay as a reference baseline when testing other designs, and treat summaries as lossy.
| Approach | Main strength | Main cost or risk | Typical fit |
|---|---|---|---|
| Auto-injected curated layers | Continuity without an explicit user action | Token cost on every call; less user control; possible mixing of unrelated contexts (Microsoft Memory Architecture Patterns) | Assistants where a small, stable profile should always be present |
| On-demand retrieval | Injects only what the current request needs | Answers depend on indexing and ranking quality | Large histories with varied topics |
| Structured or extracted memory | Inspectable, updatable facts and state | Requires extraction logic and conflict-resolution rules | Task agents, preferences, changing records |
| Full-context replay | Keeps all raw detail available | Consumes context quickly; costly and slow at scale | Short histories and baseline testing |
| Summaries | Compact and inexpensive per call | Lossy; summaries can contain hallucinated statements (Microsoft Memory Architecture Patterns) | Long-running chats where detail matters less than gist |
What published benchmark figures do and do not show
Benchmarks help you see which failure modes exist. They do not predict how your application will perform. Each figure below describes a specific system, dataset, and evaluation setup.
| Figure | Source and date | What it measured | Qualification |
|---|---|---|---|
| 500 curated questions | LongMemEval, ICLR 2025 | Long-term interactive memory of chat assistants | Size of the benchmark, not a result |
| 30% accuracy drop when memorizing across sustained interactions | LongMemEval, ICLR 2025 | Evaluated commercial chat assistants and long-context LLMs | The benchmark’s reported finding for its own evaluation; it is not a universal loss rate for all LLMs |
| 97.2% retention precision and 58% store reduction | Microsoft Research, May 2026 | Deduplication-based consolidation on a VSCode issue-tracking dataset | Reported for one method on one dataset |
| 86.3% LLM-judge accuracy on LoCoMo; 87.4% on LongMemEval | Microsoft Research, 2026 (Memora) | Memora’s own evaluation | Publisher-reported; the LLM-judge method and the comparison baselines matter for interpretation |
| Up to 98% fewer context tokens than full-context inference | Microsoft Research, 2026 (Memora) | Token use compared with full-context inference in the tested system | Applies to the tested system and comparisons only; do not expect the same reduction in your product |
None of these figures is independent validation of a product. They are one group’s reported methods and results, and they show what is possible under particular data and evaluation conditions. The Microsoft Research May 2026 paper describes six mechanisms, including sleep-phase consolidation, interference-based forgetting, and hybrid multi-cue retrieval, and it reports separate LongMemEval results. Treat the mechanisms as design ideas to test, not settings to copy.
What to measure in your own application
Build a test set from the conversations and tasks your users actually run, then check each of these abilities. LongMemEval names five, and they are a useful starting taxonomy rather than a complete production checklist:
- Information extraction: Does the system store the facts that matter, in a form that can be retrieved later?
- Multi-session reasoning: Can it combine facts from separate sessions to answer a question?
- Temporal reasoning: Does it respect when something was said or happened?
- Knowledge updates: When a user corrects a fact, do later answers use the correction and not the old value?
- Abstention: When the memory store has no evidence, does the model say so instead of inventing an answer?
Measure retrieval and generation separately. If the right item never reaches the prompt, changing the model will not fix the answer. Also record token count and latency per request, because a memory design that scores well and doubles prompt size may be unacceptable.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallA reference architecture, with example components
AWS Prescriptive Guidance gives one concrete mapping of roles to services. These are examples from that guidance, not a required stack or a neutral product comparison. Substitute equivalents from your own environment.
| Role in the memory loop | Example components named in AWS guidance |
|---|---|
| Recent state | DynamoDB, Redis, or Bedrock context |
| Structured long-term memory | Aurora, DynamoDB, or Neptune |
| Semantic retrieval | OpenSearch or Pinecone |
| Transcripts and files | S3 |
| Orchestration | Lambda or Step Functions |
| Reasoning | Bedrock |
Microsoft Research’s Memora work takes a different shape: it separates rich memory content from lightweight retrieval abstractions and cue anchors, then retrieves iteratively under a policy. That is a research system, and it shows that the storage layer and the retrieval layer can be designed separately.
Troubleshooting continuity problems
Start from the symptom and check the lifecycle step most likely responsible.
- Nothing carries over between sessions. Check the write path first. Confirm that the store receives records after each response, and that the identifier used to write matches the one used to read, such as user ID rather than session ID.
- The assistant repeats an old value after a correction. Look at the update step. The new value should overwrite or supersede the old one, with a timestamp, so the read step can prefer it.
- Unrelated details appear in answers. Inspect the retrieval filters. Queries should be scoped to the user, tenant, or task before any ranking happens.
- A confident recollection cannot be traced to anything the user said. Store a source reference with each extracted fact and check summaries against the original transcript.
- The answer omits a fact you know is stored. Run retrieval alone on that question. If the item is not returned, fix indexing or ranking; if it is returned but ignored, fix the prompt assembly or the instructions around the memory block.
- Latency or token cost has grown with usage. Check whether full history is being injected. Cap the memory block size and retrieve only what the current input needs.
In each case, the model is usually answering correctly from the context it was given. The fix belongs to the application.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
