More memory does not automatically make an AI agent smarter. Long histories can lower answer quality when retrieval returns irrelevant, duplicated, stale or conflicting facts. The useful question is not how many tokens or records an agent stores, but whether it selects the right evidence, respects time, updates contradictions, declines unsupported answers and forgets what should no longer influence a decision.
Three benchmarks cover different parts of that problem: LongMemEval tests long-term conversational reliability, MemBench measures factual and reflective memory across interaction settings, and MemoryAgentBench tests agents that learn incrementally and selectively forget. Treat retrieval scores as retrieval scores—not as proof that the final answer is correct—and hold the model, corpus, judge, retrieval depth, token budget and software version constant when comparing systems.
As an Amazon Associate I earn from qualifying purchases.
What agent memory benchmarks actually measure
An agent’s context window is the text it can read in one prompt. Memory is a longer-lived system that decides what to retain, how to represent it, when to retrieve it and whether an old item should be revised or suppressed. A larger context window may let a model read more text, but it does not decide which facts matter or whether two statements conflict.
A serious benchmark therefore evaluates a pipeline, not just a database. The pipeline usually includes ingestion, indexing, retrieval, reranking, prompt assembly, answer generation and, sometimes, a write-back or forgetting operation. Scores can change when any of those stages changes. An embedding model, chunking rule, retrieval depth or judge prompt can matter as much as the storage engine.
#1 Best Overall
- Read Before You Buy — No Video Output: These adapters support charging and USB 2.0 data transfer, but cannot transmit video signals. Except for standard USB webcams (which use USB data only), they are not compatible with HDMI/DisplayPort cables, video-capable USB-C hubs, or docking stations with video output.
- Convert USB-A Ports to USB-C: Designed to connect USB-C earphones, cables, flash drives, card readers, and other USB-C accessories to standard USB-A ports. Plug-and-play with no drivers or software required.
- Aluminum Alloy Housing: Built with a sturdy aluminum alloy shell that aids in heat dissipation and protects against daily wear and scratches. Designed to maintain a stable and secure connection.
- Compact & Travel-Friendly: The ultra-compact design allows the adapter to stay plugged into your device without blocking adjacent ports or adding bulk, reducing wear and tear on your original USB ports.
- 12-Month Warranty: Backed by a 12-month manufacturer warranty for peace of mind. Designed to meet strict quality control standards for reliable everyday performance.
Why adding more memory can make an agent worse
LongMemEval provides the clearest warning. Its authors reported that commercial assistants and long-context models lost about 30% accuracy on sustained-interaction memorization tasks. The failure is not that the system cannot store another conversation; it is that every additional item creates another opportunity to retrieve a plausible but irrelevant, outdated or contradictory statement.
As a memory grows, the agent must solve four control problems:
- Selection: keep durable facts and discard conversational noise.
- Representation: encode a fact with enough context, entities and time markers to distinguish similar events.
- Updating: replace or supersede a fact when the user changes a preference or a real-world value changes.
- Suppression: avoid using a low-confidence, revoked or obsolete item even if a similarity search retrieves it.
LongMemEval tested remedies including session decomposition, fact-augmented key expansion and time-aware query expansion. Those techniques point to the same conclusion: memory quality depends on indexing and retrieval policy, not storage volume alone. A system that stores everything without temporal and confidence controls can become less reliable as it becomes larger.
LongMemEval: the benchmark for sustained conversational memory
LongMemEval, introduced by Di Wu and co-authors in 2024, contains 500 curated questions covering five abilities:
Rank #2
- 5-in-1 USB-C Hub: Experience comprehensive connectivity featuring a Power Delivery input, two USB-A 2.0 ports, a USB-A 3.0 port, and an HDMI port. (Note: The USB-C power delivery input port is only for connecting an external wall charger to power your laptop and cannot power peripheral devices.)
- 90W Pass-Through Charging: Achieve optimal charging with 90W pass-through power to your laptop, supported by a total input of 100W, with the hub reserving 10W for operational efficiency. (Note: Wall charger not included.)
- Quick Data Transfers: Accelerate your productivity with rapid data transfers using a high-speed 5Gbps USB 3.0 port and two 480Mbps USB 2.0 ports.
- 4K HDMI Display: Enhance your visual experience with a hub capable of delivering 4K resolution at 30Hz in both mirror and extend modes. Please note that this hub is compatible with MacBook (macOS 12 and newer), Windows 10 and 11, ChromeOS, and laptops equipped with DP Alt Mode and Power Delivery. Note: This device is not compatible with Linux.
- What You Get: Anker USB-C Hub (5-in-1, 4K HDMI), welcome guide, 18-month warranty, and our friendly customer service.
- Information extraction: recover a specific fact from prior interactions.
- Multi-session reasoning: combine evidence spread across separate conversations.
- Temporal reasoning: determine which event happened first, most recently or within a specified period.
- Knowledge updates: use a newer statement when it supersedes an older one.
- Abstention: say that the available history does not support an answer.
Its sustained-interaction result is especially useful for diagnosing memory bloat. A model can perform well on a short retrieval test yet fail after many sessions because old preferences, repeated facts and near-duplicate episodes compete for attention. LongMemEval is therefore a strong first choice when your agent is a personal assistant, support representative or other system expected to remain consistent over weeks or months.
What to record when running LongMemEval
Report the model and judge version, the complete conversation corpus, memory-write policy, retrieval depth, prompt-token limit, latency and per-session cost. A single aggregate accuracy number hides whether a system fails on updates, temporal questions or abstention. Break out all five abilities and inspect the evidence supplied to the model for every wrong answer.
MemBench: broader scenarios and memory economics
MemBench, published in ACL Findings 2025 by Haoran Tan and co-authors, evaluates memory from two complementary perspectives. Factual memory concerns explicit details such as names, dates and preferences. Reflective memory concerns higher-level conclusions an agent forms from experience, such as a user’s recurring goal or a lesson inferred from several interactions.
Free tools Windows power users keep installed
One-click scans. No signup required.
It also separates participation and observation scenarios. In participation, the agent is involved in the interaction and can decide what to record while responding. In observation, it watches or receives interaction data and must build memory without being the primary conversational partner. This distinction matters for products that ingest tickets, meeting transcripts or tool logs rather than only direct chats.
Rank #3
- Sleek 7-in-1 USB-C Hub: Features an HDMI port, two USB-A 3.0 ports, and a USB-C data port, each providing 5Gbps transfer speeds. It also includes a USB-C PD input port for charging up to 100W and dual SD and TF card slots, all in a compact design.
- Flawless 4K@60Hz Video with HDMI: Delivers exceptional clarity and smoothness with its 4K@60Hz HDMI port, making it ideal for high-definition presentations and entertainment. (Note: Only the HDMI port supports video projection; the USB-C port is for data transfer only.)
- Double Up on Efficiency: The two USB-A 3.0 ports and a USB-C port support a fast 5Gbps data rate, significantly boosting your transfer speeds and improving productivity.
- Fast and Reliable 85W Charging: Offers high-capacity, speedy charging for laptops up to 85W, so you spend less time tethered to an outlet and more time being productive.
- What You Get: Anker USB-C Hub (7-in-1), welcome guide, 18-month warranty, and our friendly customer service.
MemBench reports three metric families:
- Effectiveness: whether stored memory improves the task outcome.
- Efficiency: how much time, computation or prompt material is needed to obtain that improvement.
- Capacity: how well the system continues to function as the amount and variety of information increase.
Use MemBench when you need coverage beyond question answering from a transcript. It can expose a system that retrieves facts accurately but fails to form useful reflections, or one that performs well while actively participating yet mishandles passively collected data.
MemoryAgentBench: incremental learning and selective forgetting
MemoryAgentBench is designed for agents that accumulate information over many turns instead of receiving a fixed archive at test time. Its incremental multi-turn format evaluates four competencies:
- Accurate retrieval: find the right evidence from accumulated memory.
- Test-time learning: incorporate new information during deployment rather than relying only on pretraining.
- Long-range understanding: connect events separated by many interactions.
- Selective forgetting: remove or stop using information that should no longer affect behavior.
The authors report that current methods do not master all four abilities simultaneously. That finding is important for teams tempted to optimize one leaderboard number. A memory system may retrieve old facts well while failing to learn a correction, or it may update successfully while retaining sensitive information that a user asked it to forget.
Recommended Free Tools
Why selective forgetting is a separate capability
Deleting a record is not always enough. Copies may remain in summaries, vector indexes, caches or generated profiles. A useful forgetting test checks that the target item no longer appears in retrieval results, derived summaries are corrected, and the agent does not reconstruct the deleted fact from another stored artifact. MemoryAgentBench’s framing makes forgetting an observable behavior rather than a storage-maintenance checkbox.
Rank #4
- Dual Converters, Infinite Potential：Includes 2× USB C male to USB A female adapters and 2× USB A male to USB C female adapters. Perfect for a wide range of uses—tablets with Bluetooth keyboards, expand USB ports on macbook, and more. Two different converters for all your daily needs
- Next-Level 10Gbps & 3A Charging: No more slow 480Mbps, this usb to usb c adapter has a transfer speed of up to 10Gbps, allowing you to do more transferring in less time. This usb adapter fits both USB A and USB C charger, supporting up to 3A fast charging
- Upgraded Exquisite Craftsmanship: With an aluminum alloy housing and metal connector, the usbc to usb adapter is extremely durable and sturdy. Rigorously tested to withstand more than 10,000 times of plugging and unplugging, ensuring long-lasting performance
- Broad Compatible: The usb c to usb adapter widely supports all USB C/ USB A devices like laptops, tablets, cellphones, car chargers, and phone chargers. Such as compatible with MacBook Pro/Air 2023/2022, Thunderbolt 4/3 Devices,Apple MagSafe Watch 9/8/7/SE/Ultra, iPad Pro 2022/2021, Samsung Galaxy S23/S20/S10, and iPhone 17/16/15 Pro. Plug and play
- Please Note: To reach 10Gbps speed, keep the cable under 3.3 ft. For USB A Male to USB C adapters, try flipping the USB C connector. USB C Male to USB A adapters support bidirectional 10Gbps transfer within 3.3 ft
Retrieval scores are not answer scores
A public LongMemEval-S harness measured 95.2% R@5 for a hybrid of BM25 and vector retrieval, compared with 86.2% R@5 for BM25 alone. The figures were reported by the agentmemory benchmark maintainers and accessed in 2026. R@5 means that a relevant item appeared in the top five retrieved results; it does not mean the model gave a correct, grounded or safe answer.
Keep retrieval evaluation separate from end-to-end evaluation. A retriever can place the right passage at rank five while the generator ignores it, blends it with a stale passage or answers a question that should have triggered abstention. Conversely, a model might answer correctly from a concise summary even when the original evidence was not in the top five. Record both stages and retain the retrieved evidence for audit.
Comparison of the three major benchmarks
| Benchmark | Primary scope | Memory competencies | Scale or format | Best use |
|---|---|---|---|---|
| LongMemEval | Long-term conversational reliability | Extraction, cross-session reasoning, temporal reasoning, updates, abstention | 500 curated questions | Assistants expected to remain consistent over sustained interactions |
| MemBench | Broad factual and reflective memory | Effectiveness, efficiency and capacity across participation and observation | Dataset with multiple memory scenarios; exact item count not stated | Comparing memory behavior across active and passive data collection |
| MemoryAgentBench | Incremental multi-turn agents | Accurate retrieval, test-time learning, long-range understanding, selective forgetting | Incremental multi-turn evaluation; exact item count not stated | Agents that continually learn, revise and forget |
No benchmark is a universal ranking. LongMemEval gives the strongest view of temporal updates and abstention, MemBench broadens scenario coverage, and MemoryAgentBench stresses continual learning and forgetting. For a production decision, run the benchmark that matches your failure risk and add a small private set of incidents from your own domain.
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallMetrics that make two memory systems comparable
Publish a metric sheet before testing so a favorable score cannot be obtained by changing the rules halfway through. Include:
Best Value
- 5-in-1 Connectivity: Equipped with a 4K HDMI port, a 5 Gbps USB-C data port, two 5 Gbps USB-A ports, and a USB C 100W PD-IN port. Note: The USB C 100W PD-IN port supports only charging and does not support data transfer devices such as headphones or speakers.
- Powerful Pass-Through Charging: Supports up to 85W pass-through charging so you can power up your laptop while you use the hub. Note: Pass-through charging requires a charger (not included). Note: To achieve full power for iPad, we recommend using a 45W wall charger.
- Transfer Files in Seconds: Move files to and from your laptop at speeds of up to 5 Gbps via the USB-C and USB-A data ports. Note: The USB C 5Gbps Data port does not support video output.
- HD Display: Connect to the HDMI port to stream or mirror content to an external monitor in resolutions of up to 4K@30Hz. Note: The USB-C ports do not support video output.
- What You Get: Anker 332 USB-C Hub (5-in-1), welcome guide, our worry-free 18-month warranty, and friendly customer service.
- Answer accuracy and evidence grounding: correctness plus whether the answer is supported by retrieved items.
- Retrieval recall and ranking: R@K, mean reciprocal rank or another clearly defined ranking measure.
- Temporal and contradiction handling: accuracy when a newer fact supersedes an older one.
- Abstention quality: correct refusals on questions with missing or conflicting evidence, including false-abstention rate.
- Selective forgetting: success at removing a target item without damaging unrelated memory.
- Latency: ingestion, retrieval and complete answer latency, reported separately and at useful percentiles.
- Prompt-token use: tokens inserted into the model context, not just tokens stored on disk.
- Infrastructure cost: storage, embedding, indexing, query and model costs under a stated workload.
- Reproducibility: model, corpus, judge, retrieval depth, software version, random seeds and configuration.
A reproducible evaluation procedure
- Freeze the task set. Use the published benchmark version and record any permitted adaptations. Do not remove hard questions after seeing errors.
- Define the memory policy. Document what is written, when summaries are created, how confidence is assigned, and how corrections or deletion requests propagate.
- Fix retrieval conditions. Keep BM25 or vector settings, hybrid weights, reranking, top-K and filters constant across systems.
- Fix generation conditions. Use the same model version, system prompt, temperature, maximum output and context budget wherever the comparison permits.
- Capture evidence. Save retrieved passages, ranks, timestamps, final answers and abstentions so failures can be classified rather than guessed at.
- Repeat for variance. Run enough trials to expose stochastic differences and report confidence intervals or at least run counts.
- Audit cost and latency. Measure warm and cold paths, ingestion overhead and peak behavior; a high-accuracy system that misses the product’s latency budget may not be deployable.
How benchmark findings should change an agent architecture
Write fewer, richer memories
Store atomic facts with an entity, timestamp, source and confidence instead of dumping entire transcripts into a single undifferentiated index. Keep the original evidence available for audit, but retrieve a compact representation first and expand to the source only when needed.
Make time and updates explicit
Represent validity intervals or at least event and recording times. When a user changes a preference, mark the previous value as superseded rather than leaving two equally authoritative vectors. Query expansion that includes dates and session information can reduce collisions between similar events.
Use layered retrieval
A practical design often combines lexical search for exact names and identifiers with vector search for paraphrases, followed by reranking and a temporal or policy filter. The 95.2% versus 86.2% R@5 result illustrates why hybrid retrieval can improve recall, but the final answer still needs grounding and conflict checks.
Give abstention and forgetting first-class paths
If no item clears a relevance or confidence threshold, the agent should state that its memory is insufficient. A deletion request should trigger removal from primary records, summaries, indexes, caches and derived profiles, followed by a verification query. These controls reduce confident errors and make compliance behavior testable.
Costs, vendor claims and reproducibility limits
Published scores are setup-dependent. Redis Applied AI Research reported a reproduced task-averaged LongMemEval accuracy of 86.5% for its Redis Remis plus Instruct configuration, at approximately $0.07 per session, in 2026. Those figures are a reproduced vendor report under that configuration, not a universal price or accuracy guarantee.
The same analysis found that reproduced results differed from published numbers by +3 to −28 percentage points. Such variation can come from model revisions, prompts, judges, retrieval depth, corpus preparation or infrastructure. Treat any single leaderboard result as conditional evidence, and publish enough configuration detail for another team to repeat it.
Which benchmark should you choose?
- Choose LongMemEval first for a conversational assistant where stale preferences, temporal order and unsupported answers are the main risks.
- Add MemBench when the system must build both explicit facts and reflections, or when it ingests observed conversations, tickets or logs.
- Add MemoryAgentBench when the product learns continuously and must demonstrate reliable updates and selective forgetting.
- Always add retrieval diagnostics such as R@K and rank distributions, but never substitute them for end-to-end answer and abstention tests.
The central lesson across all three benchmarks is operational: memory is a control problem. An agent earns trust by retaining the right information, retrieving it at the right time, updating it when reality changes and leaving it out when evidence is weak. Increasing the memory store without improving those controls can increase the number of ways the agent is wrong.
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
