To give a Vertex AI agent long-term memory, store information outside the model’s active context and retrieve it when a later interaction needs it. Use session state for the current chat, a persistent resource such as Agent Engine Memory Bank or a RAG corpus for cross-session recall, and treat model-side context or cache as temporary working data—not as your durable record.
What does “memory” mean in a Vertex AI agent?
It helps to separate three layers that are often called memory even though they have different lifetimes and jobs:
As an Amazon Associate I earn from qualifying purchases.
- Session and state: the short-term record an application uses to continue the current interaction, such as messages, tool results and workflow variables.
- Persistent memory or retrieval resources: application-managed information that can be found again in a later session, such as extracted facts in Memory Bank or source-bearing content indexed for RAG.
- Model-side context and cache: information currently available to the model or temporarily cached by a service. These mechanisms support inference or reconnection; they are not automatically a durable, governed memory store.
Google’s ADK documentation describes a session and its state as short-term memory for one chat. A configured memory service can add cross-session recall, but session state by itself should not be treated as a promise that the next session will inherit the previous one. Google’s long-context guidance likewise compares a context window to short-term memory: when the available context is limited, an application may summarize, filter, retrieve relevant material or drop older messages. A larger context window increases working capacity; it does not establish durable storage or a policy for updating facts.
Which Vertex AI memory approach should you choose?
| Need | Starting point | What it stores and retrieves | Main design trade-off |
|---|---|---|---|
| Keep messages, tool outputs and workflow variables available in the active chat | ADK session/state | Current interaction state, available to the application as it continues the session | It is short-term interaction state, not a cross-session memory policy. |
| Recall a concise, evolving set of facts extracted from conversations | Vertex AI Agent Engine Memory Bank | Generated and consolidated memories, retrieved using scope and similarity search | Extraction and consolidation create a curated representation rather than a guaranteed-true record. Plan for scope, provenance, corrections and expiration. |
| Find relevant passages from prior conversations or an existing indexed corpus | ADK’s VertexAiRagMemoryService or another RAG-backed design | Conversation or corpus content indexed for vector-based retrieval; results can include text and source details | You retain source-bearing chunks, but must interpret retrieval scores according to the configured database and metric. |
| Fit more relevant information into a request without retaining every message | Summarization, filtering or RAG | A smaller summary, selected prompt material or retrieved context | These manage what enters the current context; they do not, by themselves, define durable storage. |
These are starting points, not mutually exclusive products. An agent may keep the live conversation in session state, retrieve a concise personal memory from Memory Bank, and fetch source passages from a RAG corpus when the user asks for evidence. Compare options by what they retain, how retrieval works, whether provenance survives, how contradictions and corrections are handled, whether expiration is configured, and what infrastructure, latency and cost the application can support. The documentation cited here does not establish comparative performance or cost figures for these designs.
#1 Best Overall
Google’s ADK comparison distinguishes Memory Bank, which extracts and consolidates meaningful information, from VertexAiRagMemoryService, which stores conversations in Knowledge Engine and retrieves them by vector similarity. The latter can suit raw transcript recall or retrieval alongside other RAG-indexed content; the former suits a smaller evolving representation when the application prefers not to fetch whole transcript passages every time.
How do vector embeddings retrieve a memory?
- Represent content as vectors. An embedding model converts text into numeric vectors so semantically related text can be compared, even when the wording differs. Vertex AI RAG context retrieval accepts a text query and returns relevant contexts from indexed material.
- Apply the intended scope or corpus boundary. Retrieval searches only the resources available to the chosen service and request. In Memory Bank, the requested scope must exactly match a memory’s scope: the keys and values must match, and matching is case-sensitive.
- Rank candidates with the configured similarity method. The system compares the query representation with stored content representations and ranks candidate memories or chunks. Memory Bank similarity search compares a request with embeddings of facts within the matching scope.
- Return useful context to the agent. RAG results can include a source URI or display name, text and a score. The agent or application then decides what content to put into the model’s active context.
An embedding score is not necessarily a probability that a memory is correct. Vertex AI’s RAG API explains that score interpretation depends on the underlying vector database and metric. Under the documented cosine-distance example, a larger distance means a result is less relevant; other configurations may express similarity differently. Establish the metric and score direction before applying thresholds or displaying a score as confidence.
Vertex AI RAG supports dense and sparse hybrid ranking, with an alpha parameter controlling their weighting. That setting changes the blend of ranking signals; it does not make retrieved text true or resolve contradictions between sources. Google’s RAG quickstart uses text-embedding-005 as an example, not as a requirement for every corpus or workload. The Memory Bank API reference says its similarity search uses text-embedding-005 by default when a different embedding model has not been set.
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchPC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11What do Memory Bank scope, TTL and revisions control?
Scope is a retrieval boundary and part of the data model. Because Memory Bank requires an exact, case-sensitive match and a memory’s scope cannot be changed after it is generated or created, decide the scope keys and values before relying on the memories. For example, a user or tenant identifier can distinguish whose memories are eligible for retrieval. A typo, changed key or differently cased value can prevent a match; a scope mistake should not be treated as a relevance-ranking problem.
Memory Bank exposes configuration for memory generation, similarity search, automatic time-to-live (TTL) and whether revisions are created. TTL is configurable. If automatic TTL is not used, expiration can be managed through an individual memory’s expire_time. A TTL or expiration timestamp controls how long a stored memory remains eligible under that configuration; it does not verify that the claim was accurate while it existed. The API reference also exposes revision behavior, which teams should account for when deciding how changes to a memory are represented.
The Memory Bank API reference labels the feature Preview. Availability and release stage can vary over time and by region; check the current documentation and the region you plan to use before making a deployment decision. The documentation cited here does not establish a region-by-region availability matrix.
How should an agent track what it knows?
“Epistemic state” is a useful design lens for the application, not the name of a documented Vertex AI feature. It means keeping track of what the agent currently treats as known, what is uncertain, what source supports a claim, and when that claim could be stale. Vertex AI documentation describes memory generation, retrieval, scope, embeddings and TTL; it does not define an official epistemic-state resource or guarantee that a retrieved memory is true.
For durable claims, retain provenance and make corrections and conflicts visible rather than silently overwriting them. A source-bearing RAG chunk may preserve the text and its source reference; a generated Memory Bank fact may be more compact but should not be mistaken for a verbatim source. Where facts change over time, use an update, review or expiration policy appropriate to the claim. These are application-design recommendations, not built-in guarantees of either service.
A practical handoff from storage to the model should preserve uncertainty. For instance, distinguish a current, source-supported preference from a tentative inference or an older fact that needs confirmation. Retrieval says that content matched the query under a particular scope and ranking method; it does not certify truth, freshness or user consent to continue using that information.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What is ephemeral in Vertex AI—and what is not?
“Ephemeral” only has meaning when the specific feature and its retention behavior are named. Separate the active tokens sent to or retained in a model context, a service-side cache used for latency or reconnection, and durable resources controlled by the application. A fact being in the current prompt is not the same as being stored in Memory Bank; a cache is not the same as a searchable RAG corpus.
Google Cloud’s zero-data-retention documentation says published Gemini models cache customer inputs, outputs and derived data in memory by default to reduce latency. For that documented case, the cache is project-isolated and has a 24-hour TTL. Separately, Gemini Live API session resumption is disabled by default and must be enabled by the user on a request; the documentation describes cached prompts and outputs being retained for up to 24 hours to allow a session to resume. These are specific documented behaviors, not a blanket retention period for every Vertex AI feature or all customer data. The same page notes a Grounding with Google Maps exception to disabling storage.
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Clear out junk files and repair common Windows errorsFree Scan →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →For retention or privacy decisions, inspect the documentation for the exact model, API and feature in use, along with the application’s own storage configuration. Do not infer that disabling or expiring one layer deletes data held in another.
Quick Recap
How can you assemble a long-term memory flow?
- Keep the current interaction in session state. Store the messages, tool results and workflow variables needed to continue the active chat. Decide separately whether and how the session itself is persisted by your application.
- Choose the cross-session representation. Use Memory Bank when generated, consolidated facts fit the use case; use RAG-backed memory when the agent needs to retrieve passages from transcripts or other source content. It is reasonable to use both for different kinds of recall.
- Define boundaries before writing data. For Memory Bank, settle on exact scope keys and values, including capitalization, because scope matching is exact and scope is immutable for an existing memory. For RAG, determine which corpus content and source metadata the application will index and permit the agent to retrieve.
- Decide how claims stay accountable. Retain provenance where needed, define how users or systems correct a memory, and establish review or expiration rules for facts that can become stale. Do not use a similarity score as a truth or confidence score.
- Retrieve selectively for each turn. Use the current query to retrieve relevant memories or chunks, then place only useful material into the active model context. If the context is constrained, summarize or filter rather than assuming that old messages remain available indefinitely.
- Validate operational behavior. Test scope matching, update and expiration behavior, retrieval relevance, score interpretation, and the distinct retention settings of any cache or session-resumption feature used by the application.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
