AI coding agents repeat mistakes when a fix applies only to the current task—or when the system fails to preserve, retrieve, or correctly apply what went wrong. Useful “pain” is not a feeling: it is feedback such as a failed test, a reviewer’s correction, or a reminder not to change code. To improve later work, that feedback must become guidance the agent can use and then be checked on new tasks.
Why does AI keep making the same coding mistakes?
A coding agent is more than its language model. Its behavior also depends on the instructions it receives, the repository context it can see, its tools and execution environment, and how the system handles feedback. A mistake that looks like the model “forgot” may instead come from missing context, an instruction that was ambiguous, memory that did not persist, irrelevant stored guidance, or an objective that rewards producing a patch when the right action is to leave the code alone.
There is also a difference between correcting the current attempt and learning across tasks. After a test fails, an agent may revise code using that failure in the active conversation. That does not establish that the agent will remember the lesson after the session ends. Persistent rules, retrieved examples, and model-weight updates are distinct ways of influencing future behavior; one should not be mistaken for another.
What recurring failures look like in practice
Tang and colleagues’ 2026 study, “How Coding Agents Fail Their Users: A Large-Scale Analysis of Developer-Agent Misalignment in 20,574 Real-World Sessions,” analyzed logged IDE and command-line sessions from 1,639 repositories. In the visible misalignment episodes they validated, problems included misunderstood intent, violations of developer constraints, faulty implementation, overreach, and inaccurate reporting—not just syntax errors. The authors report that 91.49% of visible resolutions required explicit user correction. That figure describes resolutions in the study’s observed episodes, not all agent interactions; silent workarounds and unlogged failures are outside that measure.
#1 Best Overall
The same study reports that 90.50% of episodes imposed effort or trust costs rather than irreversible system damage. Those are the authors’ classifications of observed episodes, not a guarantee that coding-agent mistakes are harmless. The dataset consists of public, opt-in logs and differs in agent and task composition across IDE and CLI settings, so it should be read as evidence about logged cases, not a universal failure rate.
What “teaching it pain” actually means
Here, pain is a metaphor for a consequence that makes an error visible. A test can show that behavior is wrong; a tool error can reveal a failed operation; a code review can identify a violated convention; and a user can point out that the request was misunderstood. But a signal is useful only if the system can interpret it, connect it to the cause, and use it where a similar decision arises again.
A productive feedback loop has several distinct stages:
- Expose the failure. Use tests, execution results, review comments, or a clear user correction to show what did not work. A passing test is evidence only for the behavior that test covers; it cannot by itself establish that code is safe, maintainable, or consistent with unstated requirements.
- Identify the reusable lesson. Explain the violated requirement or pattern, not just that the output is “wrong.” For example, distinguish “this change breaks the parser test” from “do not alter this public interface without updating its callers.”
- Choose where the lesson belongs. Correct the active attempt, save a repository-specific rule, or update a broader system instruction only when the lesson really applies at that scope.
- Retrieve and apply it later. A saved rule has no effect if it is unavailable or ignored in the next relevant task. The agent needs to see the guidance at decision time.
- Check transfer and side effects. Test a later, similar task and look for both recurrence and overcorrection. A rule that prevents one bug by blocking legitimate changes has not solved the problem.
The practical aim is better decisions in relevant future situations—not a claim that the system has acquired feelings or human-like wisdom.
What can change when an agent receives feedback?
“Learning” can refer to several different mechanisms. They vary in persistence, scope, and who controls the change.
| Mechanism | What changes | When it can help | What to check |
|---|---|---|---|
| Correction in the active session | The current conversation or task context includes the correction. | The agent can revise its present attempt while the relevant context remains available. | Whether the revised code passes relevant checks; this alone does not show cross-session retention. |
| Retrieved memory or prior examples | The system makes stored experience available to a later task. | A later task resembles the earlier case and the relevant memory is retrieved. | Whether the memory is accurate, current, relevant, and actually surfaced. |
| Persistent rules or skills | An instruction set, checklist, or skill guides future work. | A recurring, well-understood lesson should influence later tasks in its intended scope. | Who approves and edits the rule, whether it fits the repository, and whether it creates overbroad behavior. |
| Model-weight updates | The model’s trained parameters are changed. | When a training process is intended to alter the model itself. | Ordinary user corrections are not established here as updating a coding agent’s weights; verify the specific system’s behavior rather than assuming it. |
These approaches are not interchangeable. A persistent prompt or version-controlled rule file can shape behavior without changing model weights. Conversely, a correction made during one session is not evidence of lasting memory.
Can review comments become reusable rules?
One proposed approach turns accepted code-review comments into persistent behavioral rules and pairs those rules with a self-review checklist. In their 2026 framework paper, “Self-Improving AI Coding Agents Through Accumulated Behavioral Rules: A Closed-Loop Framework,” Aditya Aggarwal and Nahid Farhady Ghalaty state the design principle: “Every accepted review comment is a self-review rule.” The useful idea is to convert an accepted correction into something the agent can check before submitting similar work.
The authors describe a deployment on a microservices platform with more than 35 services. They report growing the rule set from 5 to 18 behavioral rules, adding more than 15 language-specific standards, and using a 15-item self-review checklist. Their reported evaluation covered 11 recorded sessions and found a 0% recurrence rate for error classes addressed by rules. These are results from the authors’ limited deployment, not an independently replicated or general-purpose estimate of how often rule-based memory prevents mistakes.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
How to make a rule useful rather than merely memorable
- Make it specific. Record the condition that triggered the correction and the action to take, rather than a vague admonition such as “be careful.”
- Set its scope. Mark whether it applies to one repository, language, service, interface, or task type. A local convention should not silently become a universal rule.
- Keep it reviewable. Have a developer accept, edit, or remove persistent guidance. A mistaken rule can reproduce errors just as reliably as a good one can prevent them.
- Link it to a check. Where possible, translate the rule into a test, lint check, or explicit self-review question. This makes compliance easier to verify than relying on a reminder alone.
- Test for overgeneralization. Try a related task where the original constraint does not apply. Confirm that the agent can distinguish the cases instead of refusing all similar work.
Why teaching an agent not to act matters
A feedback system that rewards only completed changes may encourage unnecessary edits. Sometimes the correct response is to explain that no change is needed, ask a clarifying question, or report that the evidence is insufficient.
Rank #4
Gloaguen and colleagues’ 2026 FixedBench study tested five models across four agent harnesses on 200 human-verified tasks where no code change was required. The authors found undesirable proposed changes in 35% to 65% of those cases. An instruction to reproduce an issue before patching partly helped, but also led agents to abstain when an issue had only been partially fixed. The result illustrates why “always try a patch” and “never act without reproducing” are both blunt policies: reliable behavior requires distinguishing a real, actionable defect from a resolved, absent, or uncertain one.
For a developer, useful no-change guidance can be explicit: describe what evidence would justify a change, ask the agent to inspect the relevant behavior before editing, and tell it to report when the evidence does not support a patch. Then check that this restraint does not suppress legitimate fixes.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.How should developers judge whether an agent learned?
A benchmark pass rate can be informative, but it is not a complete account of an agent. Gorinova and colleagues’ 2026 position paper, “Position: Coding Benchmarks Are Misaligned with Agentic Software Engineering,” argues that benchmarks can fold the model, harness, and environment into one score, rely on a single reference solution, and provide too little component-level feedback for iterative improvement. That makes it hard to tell whether an apparent improvement came from the model, better tools, more repository context, or an easier environment.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Fix the driver behind crashes, sound loss and screen glitches3Repair Windows errors before they cause bigger problemsBest Value
When comparing systems or evaluating a feedback process, look beyond whether a task was marked complete:
- Test coverage: Do the checks cover the behavior and constraints that matter, including likely regressions?
- Correction retention: Does an accepted lesson remain available across sessions, and is it retrieved on a relevant task?
- Constraint-following: Does the agent respect instructions about scope, interfaces, files, and acceptable changes?
- Abstention: Can it avoid unnecessary edits or ask for clarification when evidence is weak?
- Transfer: Does the correction help in a genuinely similar new task without spreading to cases where it does not apply?
- System effects: Are the model, harness, tools, and environment considered separately where possible?
- Safety and maintainability: Does the proposed change preserve properties that the tests may not measure?
A survey by Zhou and colleagues, “Self-Evolving Coding Agents” (2026), describes systems that adapt memory, skills, tools, frameworks, models, or collaboration structures from prior interactions. It also identifies unresolved challenges including feedback reliability, benchmark overfitting, safety, maintainability, cost, and generalization. Adaptation is therefore not automatically improvement: the signal must be trustworthy, and changes to an agent need checks for the problems they might introduce.
What human feedback can—and cannot—prove
A 2024 preprint, “Can Language Models Solve Olympiad Programming?”, reports a tutoring experiment involving 15 programming problems. Both GPT-3.5 and GPT-4 initially had a zero solve rate in that setup. After human feedback, GPT-4 solved 13 of the 15 problems (86.7%), while GPT-3.5 remained at zero. This small, task-specific result shows that feedback can affect models differently under a particular tutoring setup; it is not a general success rate for today’s coding agents or evidence that ordinary corrections will reliably transfer to other repositories.
Feedback also affects the person delegating work. Mehra and colleagues’ 2026 paper, “Agents That Teach: Towards Designing Incidental Learning Back into AI-Assisted Software Development,” argues that delegating coding can remove some incidental learning developers get from solving problems themselves. The authors propose design principles and a SHIELD system concept for surfacing learning moments. This is a research argument and proposal, not proof that AI assistance causes skill loss or that the proposed system prevents it. For developers who want to retain understanding, asking an agent to explain a change, identify assumptions, or show the relevant tests can make its work easier to review—but those explanations still need verification against the code and behavior.
A practical way to stop a mistake from recurring
- Describe the failure precisely. Record the observed behavior, expected behavior, and the constraint or requirement that was missed.
- Find the right level for the fix. Correct the current code if the defect is local; improve the task instructions if they were ambiguous; add a repository rule if the lesson recurs and has a clear scope.
- Turn the correction into an observable check. Add or update a test when the behavior can be tested, and include a concise self-review question for constraints that are not automated.
- Make later tasks see the guidance. Confirm that the agent has the relevant repository instructions or retrieves the stored rule. Do not assume persistence merely because the agent acknowledged the correction.
- Evaluate both sides of the outcome. Check whether the original error returns and whether the rule blocks valid changes or encourages unnecessary abstention.
If the mistake returns, inspect the entire chain: Was the cause correctly diagnosed? Was the rule saved at a useful scope? Did the next task include it? Could the agent follow it alongside conflicting instructions? Did the test actually cover the failure? Each answer points to a different remedy; simply repeating the correction may not address the underlying gap.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
