Vector search is useful for recalling information by meaning, but it is not a complete agent-memory system. Production memory needs scoped, traceable records; retrieval that can handle exact terms as well as semantic matches; explicit rules for updates and deletion; and evaluations based on the questions the agent must answer. Add graph retrieval or reranking only when measured workloads justify their added complexity, latency, and cost.
Why isn’t vector search enough for agent memory?
Embedding-based retrieval is good at finding conceptually related passages even when a user phrases a question differently from the stored information. But semantic similarity is not the same as an exact match. A model may not represent a newly introduced product name, arbitrary product number, internal codename, or identifier in a way that reliably retrieves it.
As an Amazon Associate I earn from qualifying purchases.
Token-based retrieval complements embeddings by matching the words and terms actually present in the query. Google Cloud describes hybrid search as a combination of semantic and token-based methods, and names TF-IDF, BM25, and SPLADE as sparse-retrieval approaches. The right choice depends on your corpus and query patterns; hybrid search is not automatically better simply because it uses more than one retriever. Google Cloud’s hybrid search documentation
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →There is also a more fundamental limitation: ranking cannot make an ineligible record safe or relevant. A highly similar memory from another tenant, a superseded fact, or an expired session should not enter the candidate set just because its embedding scores well. Scope and lifecycle rules belong in the design alongside retrieval.
What should an agent memory record contain?
Start with the record and its lifecycle, not the index. Store durable memories as typed, scoped records rather than as anonymous text chunks. A practical record should preserve enough context to decide who may retrieve it, when it is relevant, and where it came from.
- Identity and scope: a stable record ID, owner or tenant, source, and memory type, such as user preference, project fact, or session summary.
- Content and provenance: the canonical fact or text, plus a pointer to the originating document, event, or conversation when available. Keep derived summaries or extracted memories distinguishable from original information.
- Time and status: creation and update times, any relevant valid-time range, and lifecycle state such as active, superseded, expired, or deleted.
- Retrieval fields: searchable text and any useful structured attributes, such as entity IDs or exact identifiers.
Decide what happens when information changes: does a new record supersede an old one, update it in place, or coexist with it as a time-bounded fact? Define how expiry and deletion propagate to vector, lexical, graph, cached, and derived representations. There is no single retention policy established by the cited sources; the policy must fit your product, legal obligations, and data sensitivity.
How should retrieval work across query types?
A robust pipeline first determines which memories are eligible, then retrieves and ranks candidates using the methods suited to the query. Treat the stages as separable so each can be evaluated and removed if it does not help.
- Apply access and lifecycle constraints. Restrict records by tenant or owner, source, memory type, time range, and active status before ranking. Enforce authorization at the retrieval boundary, not only in the prompt or application display layer.
- Retrieve semantic candidates. Use vector search for paraphrases, conceptual matches, and questions whose wording differs from the stored memory.
- Retrieve lexical candidates where exact terms matter. Use a token-based method for identifiers, names, numbers, codenames, and literal strings. A query router can enable lexical search selectively, or the system can run it alongside vector retrieval when the workload warrants that cost.
- Merge candidates and inspect ordering. Rank fusion is one way to combine results from separate retrievers. Google Cloud’s GraphRAG reference architecture describes merging keyword and semantic results with reciprocal rank fusion (RRF). That is an implementation example, not a universal weighting rule. Google Cloud’s multimodal GraphRAG architecture
- Add graph traversal for relationship questions. When a question depends on links among people, projects, events, or records, use graph queries or neighbor expansion to follow those connections. A flat nearest-neighbor search may find a relevant entity without finding the relationship needed to answer the question.
- Rerank only if the improvement earns its overhead. A reranker can reorder a candidate set, but it adds a processing stage and may add model or infrastructure cost and latency. Keep it only if evaluation shows a useful improvement in final-context relevance or answer grounding.
Scope filtering is a safety and relevance boundary, not merely another ranking feature. If filtering happens only after a broad search, unauthorized or stale candidates may already have reached logs, caches, rerankers, or other components. Design the query so restricted records are excluded as early as your storage and retrieval architecture allows.
When do fusion, graphs, and reranking pay off?
Each additional retrieval mechanism should correspond to a failure mode you can name and measure. Fusion is useful when vector-only search misses exact-term results and lexical-only search misses paraphrases. Graph retrieval is useful when correct answers depend on connections. Reranking is useful when the initial candidate pool contains relevant material but orders it poorly.
A limited Oracle Developers demonstration illustrates why these are hypotheses rather than defaults. Jeremy Daly’s August 25, 2026 article tested a 23-document corpus; in that small experiment, equal-weight hybrid fusion underperformed vector-only retrieval, while reranking improved ordering at a material latency cost. This is not a general benchmark of hybrid retrieval or production architectures. Daly’s practical criterion is: “Fusion and reranking are useful only when they improve a labeled workload without admitting stale or unauthorized memory.” Oracle Developers: Hybrid Retrieval for Agent Memory
How should you evaluate the whole memory pipeline?
Build a labeled set of representative questions and the memories that should support each answer. Include both ordinary lookups and deliberate edge cases. Evaluate the candidate set and the final context separately: a relevant memory that is retrieved but dropped before generation is a different failure from one that was never found.
- Semantic paraphrases: questions that express a stored concept using different wording.
- Exact lookups: names, product or ticket numbers, identifiers, codenames, and other literal terms.
- Relationship questions: prompts that require traversing connections among entities, events, or records.
- Lifecycle cases: facts that have expired, changed, or been superseded, where the current version should win.
- Boundary cases: records from a different tenant, source, or permission scope that must not appear in candidates or context.
Compare vector-only and lexical-only retrieval with fused, graph-enhanced, or reranked variants when they match your workload. Track whether relevant memories appear among candidates and in final context; whether the generated answer is grounded in the right source and version; and the latency and cost introduced by each stage. Record ablations so you can tell which component caused an improvement or regression. The cited materials do not establish universal quality thresholds or a comparable production benchmark across memory architectures, so set acceptance criteria from your application’s risk and service requirements.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.How should memory persist across sessions?
Keep short-lived session state conceptually separate from durable user or domain memory. Session state supports the current interaction and may be disposable or reconstructable; long-term memory is deliberately retained and needs explicit ownership, provenance, and lifecycle rules. Decide which writes must complete before the agent responds, which can be processed asynchronously, and what happens if extraction or embedding fails after the original interaction has been stored.
Operational design should cover concurrent updates, index refresh, backups and restore, access controls, tenant isolation, provenance audits, and deletion propagation. Retention and expiry should be enforced in the system of record as well as in derived indexes. A persistent database is a foundation for durable state, not a guarantee that the agent’s memory is correct, authorized, or production-ready.
Which storage topology fits the workload?
The sources illustrate two broad patterns, not a head-to-head performance or cost comparison. Choose based on data shape, team capacity, security boundaries, deployment needs, and the operational burden you can support.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →| Pattern | What it can consolidate | Trade-offs to assess |
|---|---|---|
| PostgreSQL-centered | Relational application state alongside vector, full-text, and graph capabilities, depending on the extensions and managed-service support selected. | Verify required extension availability, operational limits, and whether one system can meet the workload’s isolation and scaling requirements. |
| Multi-service | Separate services for elements such as object storage, graph retrieval, session persistence, and agent orchestration. | Assess cross-service consistency, access control, backup and restore coordination, network latency, and the staffing needed to operate each component. |
Compare candidates using actual constraints: tenant and security boundaries, volume and structure of data, restore objectives, retrieval latency, deployment location, vendor dependency, and total operating cost. The cited documentation does not provide comparable pricing or workload benchmarks, so it cannot establish a universal cost or performance winner.
Quick Recap
What the platform examples establish
- Google Cloud: Its reference architecture describes multimodal records consolidated into a knowledge graph and a workflow that can select keyword, semantic, or hybrid retrieval. It names Spanner Graph and Memory Bank among the services used. This documents one Google Cloud design; it is not independent evidence that those services outperform alternatives. Architecture details
- Azure HorizonDB: Microsoft Learn describes agent memory and PostgreSQL-based vector, keyword, graph, hybrid-search, and reranking capabilities. The page labels HorizonDB as Preview and was last updated July 7, 2026; check the linked documentation for current availability and supported capabilities before depending on them. Microsoft Learn: Build AI Agents with Azure HorizonDB
- Oracle AI Database: Daly’s companion implementation uses Oracle AI Database 26ai Free to demonstrate a hybrid SQL pipeline. Its 23-document evaluation is a limited demonstration, not a production bake-off or an architecture comparison. Oracle Developers article
A practical build sequence
- Define memory policy and schema. Specify record types, ownership, provenance, update and supersession behavior, expiry, deletion, and the distinction between session and durable data.
- Implement scoped retrieval first. Enforce tenant, source, type, time, and lifecycle constraints, then establish a vector-search baseline for semantic questions.
- Add exact-term retrieval against known gaps. Use representative identifiers and names to check whether lexical retrieval improves recall without flooding the candidate set with irrelevant matches.
- Introduce fusion, graph retrieval, or reranking one at a time. Match each component to a workload pattern and evaluate its effect before making it part of the default path.
- Exercise failure and lifecycle paths. Test failed extraction or embedding jobs, concurrent updates, expiry, deletion, backup restore, and authorization boundaries across derived stores.
- Set operational acceptance criteria. Review grounding, stale-memory errors, boundary violations, latency, and cost on the labeled workload, then retain only the complexity that meets the application’s requirements.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
