An agent’s “done” message is a claim, not proof that the requested work ran or that its result was saved. Check the execution record, read the actual tool or command result, and inspect the file, message, database, or deployment the agent says it changed. The artifact or destination state—not the completion prose—is the ground truth.
What “nothing ran” can mean
A completion claim can be wrong in several different ways. In one practitioner’s recorded experiment, the agent could finish successfully without ever running the separate verification call; a tool could report a processing error while the surrounding process still exited with code zero; or a tool could accept an argument that looked plausible but was invented. The experiment also found cases where the visible answer was empty even though intended content appeared in a tool-call argument. These are observed failure shapes, not proof that every agent or framework behaves this way. SunnyDachs’s report frames the key distinction plainly: “Did the verification tool run?” is different from “Did the process succeed?”
![Claude AI for Beginners Bible: [5 in 1] The Ultimate Guide to Automate Your Work, Save Hours Every Week, and Use AI for Real-World Results](https://m.media-amazon.com/images/I/415+fSJacsL._SL160_.jpg)
As an Amazon Associate I earn from qualifying purchases.
A separate troubleshooting guide describes three other possibilities: the agent emitted a command but did not read its exit result; it treated partial completion as full success; or it lost track of an error without demonstrating that the error was resolved. These are diagnostic possibilities, not a measured taxonomy of agent failures. Grolea’s guide explains why an agent’s own confidence is not an independent check: “The agent has no external check that its claim of done matches the state of the world, so its claim is only as good as its own attention in the moment it wrote it.”
How to check whether the work actually happened
- Make the claim specific. Write down exactly what the agent says it changed, sent, tested, or published. “Finished” is too vague to verify.
- Check the execution record. Inspect the tool or wire log for the expected operation. A calm-looking log or successful process exit does not establish that a separate verification call occurred.
- Read the real result. Review command output, tool responses, errors, and exit status. For an earlier CI run, retain and inspect its evidence log rather than relying on the agent’s summary. SunnyDachs’s report discusses the distinction between a process result and verification evidence.
- Inspect the destination independently. Open the claimed file, check the sent folder, query the relevant database row, or run the deployment health check. Confirm the resulting state, not just the attempted action.
- Check scope and errors. Compare the result with the original request to catch work that was silently narrowed. For every meaningful error, look for a concrete fix and a successful rerun before accepting completion. Grolea’s troubleshooting guide recommends this distinction between claimed, partial, and resolved work.
Choose a control that matches the risk
Controls should check evidence outside the agent’s own completion message where practical. Which one makes sense depends on what evidence it checks, where the decision is made, and how costly or reversible a false completion would be.
#1 Best Overall
- Small automated pipelines: Assert that the expected deliverable exists and is non-empty.
- Coding tasks: Require the expected verification command to run and gate completion on its actual result, not on a generated summary.
- Tool integrations: Validate inputs against defined types, enums, or known values so an accepted but invented argument cannot masquerade as success.
- Consequential side effects: For actions such as writing important data, billing, or publishing, separate approval from execution and consider keeping an audit record at the tool boundary.
These are practical controls recommended in SunnyDachs’s account; the reported experiment does not establish how effective they are relative to one another.
Use a CI gate for coding-agent claims
cngx’s guide documents a pattern for checking a coding agent’s claim against a real command or an existing evidence log. The verifier takes the claim and either runs a command after -- or parses a log, then compares the claim with the result. In the guide’s convention, exit code 0 means verified, 1 means blocked, and 2 means usage error. Its parser list includes pytest, unittest, jest/vitest, go test, cargo test, rspec, phpunit, dotnet test, mocha, Maven Surefire, and a generic exit-code fallback. The guide’s GitHub Actions example uses cngx v0.2.0; check the project’s current version and behavior before adopting that implementation.
Use review or approval for higher-impact work
Grolea’s Paperclip guide describes separate review and approval stages before a task is marked done: review checks the completed work, while approval requires explicit sign-off. This is vendor guidance for its own platform, not an independent comparison. Its useful general distinction is between checking the work and authorizing the outcome. Calibrate those gates to the consequence of a false completion rather than adding approval overhead to every low-risk task.
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →What the available observations do—and do not—show
SunnyDachs’s 2026 report describes three runs per test cell and explicitly calls the results “A trend, not a statistical claim.” It used one model, varied sampling conditions, did not use identical parameter descriptions across frameworks, and simulated the irreversible operation. Those observations illustrate ways verification can fail; they do not support a general failure rate, a ranking of agent frameworks, or a claim about how often this happens in production. No broad prevalence statistic or independently validated framework ranking is established by the cited material.
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
