An incident response agent that learns from past incidents needs a memory layer that stores outcomes, not chat transcripts: the symptoms, the root cause, the steps that worked, the steps that failed, and the pitfalls. It then retrieves those episodes next to current runbooks and live telemetry, and treats them as evidence to test rather than instructions to run. Microsoft’s guidance on agent memory states the principle in one sentence: “Memory is candidate context, not authoritative truth.”
This article lays out a design for that memory layer for SRE, platform, and security engineers. It draws on the documented behavior of Azure SRE Agent, AWS DevOps Agent and AWS Security Incident Response, Microsoft’s memory-safety guidance, a Google Security Blog post on generative AI in incident response, and one 2026 research preprint. It is an architecture write-up based on those public descriptions. It does not report a build or a benchmark run by the author.
As an Amazon Associate I earn from qualifying purchases.
What the agent’s memory has to hold
Raw conversation history is a poor incident memory. It is long, noisy, and full of dead ends that look the same as the eventual fix. Azure SRE Agent’s memory documentation describes a different approach: after an investigation it captures structured fields, namely symptoms, root cause, steps that succeeded, and pitfalls, and exposes them as searchable session insights. Its examples also describe saving failed strategies and configuration gotchas, which matter as much as the final fix. Microsoft frames the payoff this way: “Your agent becomes more effective over time by remembering what worked in past incidents and referencing your documentation.” That is vendor positioning, not an independently measured result.
A practical episode record, written when an incident closes, looks like this. The field list is an illustrative design assembled from the capabilities the vendors document, not any product’s schema.
| Field | What it contains | Why it earns its place |
|---|---|---|
| Identity and scope | Service, resource, environment, tenant, and an incident fingerprint (alert name plus key dimensions) | Lets retrieval prefer history for the exact resource and keeps one tenant’s data out of another’s results |
| Time window and versions | Start and end times, software and configuration versions, recent deployments | The main input to a later freshness check |
| Symptoms and evidence | Observed signals, with links to the original logs, metrics, and traces | Gives the responder something to verify instead of a summary to trust |
| Root-cause hypothesis and confidence | The cause as concluded, how certain it was, and what ruled out alternatives | Separates a confirmed cause from a plausible guess |
| Actions tried and outcomes | Each step, who or what ran it, and whether it worked, failed, or made things worse | Failed and harmful steps are the most valuable warnings |
| Verified resolution | The evidence that confirmed recovery | Distinguishes “we did X and it went quiet” from “X fixed it” |
| Applicability conditions and caveats | When this fix does and does not apply | Prevents blind reuse |
| Source links | Tickets, runbooks, postmortems, deployment changes | Makes every retrieved item traceable |
| Provenance | Who or what wrote the entry, when, and with which model version | Needed for audit, correction, and rollback of bad memories |
The five-stage loop
1. Capture
Run capture at incident close, not continuously. Convert the investigation into the episode record above, keep unsuccessful actions and caveats, and have a human confirm the root cause field when the stakes justify it. An agent that writes its own unreviewed conclusions into its own memory can compound a wrong diagnosis.
2. Index with boundaries
Scope each memory to the relevant service, resource, tenant, or incident fingerprint, and keep timestamps, source identity, and model or version provenance on every entry. Azure’s documented example prioritizes history for the exact resource involved. Microsoft’s safety guidance goes further and recommends deterministic isolation by user, agent, and tenant, with provenance recorded on each entry. In other words, isolation belongs in the data layer’s access controls, not in a prompt that asks the model to behave.
The same guidance and AWS’s DevOps Agent documentation point to a second axis: history grouped by what it describes. AWS documents per-monitor histories and recurring-cause tracking, so a pattern like “this alert keeps firing for the same reason” is visible as a pattern.
Free tools Windows power users keep installed
One-click scans. No signup required.
3. Retrieve
Search three things together: prior incidents, maintained documentation such as runbooks, and current observations. Azure documents retrieval from past incidents, user memory, and a knowledge base, and its incident workflow correlates monitoring data and deployment history where those sources are connected. A single vector search over old tickets covers only one of the three.
What comes back matters as much as what is searched. Each hit should carry the source incident, the matching evidence, what succeeded and failed, and when it happened. A well-formed result reads like this:
Similar incident: checkout-api latency, 2026-03-14 (resource: checkout-api, prod-eu)
Matching evidence: p99 latency up, connection pool saturation, deploy 40 min earlier
Root cause (confirmed by on-call): pool size regression in config v212
Worked: rollback to config v211
Failed: restarting pods (recovered for ~10 minutes, then recurred)
Caveat: only applies if the pool setting is present in the running config
Current check: running config is v230. Verify the pool setting before reusing.The last line is the point. The agent should not convert an old resolution into an instruction. It should present it as a hypothesis with a check attached.
4. Reason and act
Have the agent form testable hypotheses, gather current evidence, and compare it with the historical case. Recommend the least risky diagnostic or remediation step first. Keep read-only investigation separate from state-changing actions, and put writes and destructive operations behind explicit policy or human approval. Azure SRE Agent documents configurable run modes, with the agent either proposing actions for approval or resolving autonomously, depending on the mode. That is the shape to copy: autonomy as a setting you can tighten per environment, not a fixed behavior.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
5. Close the loop
After the incident, record whether the action worked, what evidence confirmed it, and whether the root cause changed from the original diagnosis. Let responders correct prior memory. AWS documents reflections derived from investigation feedback and learning from prior investigations, and describes memory items that expire or are refreshed. A memory that cannot be corrected or aged out becomes a liability within a few release cycles.
Where this goes wrong, and what to do about it
| Failure | What happens | Safeguard |
|---|---|---|
| Stale fix | The environment or software version changed since the memory was written | Store timestamps and applicability conditions. Check current telemetry and deployment context before reuse. Expire or refresh items, as AWS documents |
| Wrong match | Superficially similar symptoms have a different cause this time | Return the evidence and the alternative outcomes with each hit, then test a hypothesis instead of copying a fix |
| Memory poisoning | Untrusted content, such as text in a log line or ticket, persists and shapes later behavior | Gate memory writes by authorization and intent, sanitize or block sensitive or malicious content, re-evaluate content at retrieval, and make memory influence visible to the responder |
| Cross-tenant disclosure | One customer’s or team’s incident details surface in another’s session | Deterministic access controls and scoped identity. Prompting alone is not a boundary |
| Untraceable actions | Nobody can reconstruct why the agent did something | Log memory create, read, update, and delete operations and agent actions with source and identity. Keep enough history to reconstruct the incident and roll back |
| Privacy and cost | Retention and logging create storage, privacy, and latency overhead | Set retention deliberately. Microsoft explicitly names retention, logging volume, and retrieval-time safety checks as trade-offs to budget for |
The poisoning row deserves emphasis for incident work specifically. Incident data is full of text the responder did not write: error messages, user-supplied strings, third-party payloads. Anything an agent summarizes into memory can later be retrieved as if it were trusted guidance. Microsoft’s safety guidance recommends lifecycle logging and retrieval-time safety checks for exactly this reason.
Operational versus security incidents
The memory pattern is the same, but the stakes of the action step differ. For reliability incidents the useful memory is mostly remediation history: which rollback, scaling change, or configuration fix resolved a given service’s failure mode. For security incidents, memory more often holds investigation patterns and evidence, and the risk of an automated state change, such as isolating a host or revoking credentials, is higher.
AWS’s AI investigative agent for Security Incident Response illustrates a conservative stance. It gathers evidence with read-only permissions, logs its accesses to CloudTrail, and is limited to cases AWS supports. If you are comparing it with a self-built agent, that eligibility limit is a real boundary on where it applies. Azure’s incident workflow, by contrast, is documented around integrated monitoring and deployment data and configurable run modes (the page was last updated 2026-03-27).
What the evidence does and doesn’t show
No broadly applicable, independently validated statistic exists in the sources reviewed for how much an AI incident response agent improves outcomes in production. Treat any general claim of faster resolution drawn from vendor descriptions as unsupported. Vendor documentation tells you what a product does, not that autonomous response is safer or faster.
Google’s Security Blog post on generative AI in incident response (April 2024) describes using it in the response workflow and also describes summary-quality problems the team encountered. That is a useful caution for any capture stage that relies on a model to summarize an incident: summaries need review, and they need links back to source material.
The one quantitative research result in the material is a 2026 arXiv preprint, Incident Memory: Training-Free Operational Memory through Sequential Pattern Mining and Velocity-Stratified Retrieval. Its figures are author-reported and tied to a specific dataset and controlled setup:
- The data: 141,712 events across 24,918 incidents from the UCI ITSM event log, which yielded 23,110 ordered traces and 39 mined playbooks.
- Coverage: 84.3% of 6,934 held-out incidents.
- Precision: 99.2% ordered playbook precision on controlled benchmarks.
- Staleness: 36% stale returns for a flat baseline, which is the problem that freshness-aware retrieval is meant to address.
These numbers show that mining ordered action sequences and weighting retrieval by recency can beat flat retrieval on that data. They are not a promise about your environment, and a preprint has not gone through the review a journal paper has.
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Build or buy: eight questions to ask
Whether you assemble this yourself or adopt a managed agent, compare options on the same axes:
Quick Recap
- Memory representation: does it capture outcomes, failures, and caveats, or only summaries?
- Retrieval scope and freshness: is retrieval scoped to the resource, and do items expire or refresh?
- Provenance and citations: can every recommendation be traced to a source incident?
- Integrations: alerts, logs, metrics, deployments, tickets, and runbooks. A memory without live telemetry has nothing to check itself against.
- Isolation: are tenant and resource boundaries enforced by access control rather than prompts?
- Permissions and approvals: is read-only investigation separable from state-changing actions, with approval gates?
- Audit and correction: are memory operations and actions logged, and can responders correct bad memories?
- Eligibility: which clouds and case types are supported? AWS’s security investigative agent, for example, covers only supported AWS cases.
A rollout order that limits risk
- Start with capture and search only. Have the agent write episode records and let responders query them, with no automated actions.
- Add retrieval-time checks: relevance, freshness, and a visible link to the source incident on every suggestion.
- Let the agent propose remediation, with a human approving each state-changing step.
- Enable autonomous action only for narrow, reversible, well-understood cases, with logging that lets you reconstruct and undo what it did.
- Review memory regularly: expire stale entries, correct wrong root causes, and audit what was retrieved when an agent’s suggestion turned out to be wrong.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
