DriversRecommendedOutdated drivers can make a good PC feel brokenScan driver issues before chasing fixes manually.Scan NowHalloween DealsAmazon USHalloween week deals: useful picks before month-endAmazon US: current deals, useful picks and tech finds.Check DealsSlow PC?RecommendedPC slow today? Run a repair scan before it gets worseResolve common Windows issues and optimize system performance.Scan Now×
Skip to content
World desk5 min

Your Agent Says the Job Is Done. Who Verified It?

A completion message is only a claim. Verify the intended system state, assess process and policy separately, and check how reliable the verifier itself is.

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

The agent’s completion message is a claim, not proof. To verify the work, check whether the requested result exists in the relevant system, whether it meets criteria drawn from the user’s actual request, and whether the agent respected any process, consent, or safety constraints. Those checks answer different questions: a task can reach the right outcome through an unacceptable action, or the agent can follow the right process but fail to complete the task.

What counts as proof that an agent finished?

Look for evidence in the system or environment the task was meant to change. A message saying “done,” a successful tool response, or a log showing that an action was attempted may not establish that the intended result actually occurred. For example, an API call can be recorded while the target message, record, or calendar change is still absent. A check of the resulting state is stronger evidence of completion.

As an Amazon Associate I earn from qualifying purchases.

Start by turning the user’s request into specific, observable completion conditions. Keep those conditions faithful to what was asked: an evaluator should not add requirements the user never stated and then treat their absence as failure. Microsoft Research describes such invented evaluation requirements as “phantom” rubric criteria. A useful condition is independently checkable—for instance, whether the requested item appears in the correct place—not a subjective impression that the agent’s explanation sounds convincing.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Check the final state, not just the action log

  • Identify which system or record should have changed.
  • Inspect that system for the requested result, rather than relying only on the agent’s summary or an ambiguous tool response.
  • Compare the observed result with each completion condition, including whether the task is only partly complete.

Agent-Diff uses this principle in a benchmark for enterprise API tasks: it defines success by whether the expected change in environment state was achieved. Its sandboxed evaluation covered 224 tasks and nine language models across Slack, Box, Linear, and Google Calendar interfaces. Those figures describe the benchmark, not expected production reliability.

Was the result achieved in an acceptable way?

Record outcome and process separately. The outcome check asks whether a reasonable user would consider the requested task done. The process check asks whether the agent followed the relevant instructions and constraints while doing it. Microsoft Research’s computer-use work separates rubric-based process scoring from binary outcome scoring and distinguishes controllable mistakes from failures caused by the environment, such as a login wall or CAPTCHA.

This distinction makes the result easier to interpret. An agent might be blocked by an external obstacle despite making appropriate attempts; the user’s goal is still not complete. Or it might produce the requested result while taking an unauthorized side action; completion alone does not make that behavior acceptable. Report the outcome and the process finding as separate facts.

Include policy and authorization in the acceptance rule

When a task involves consent, access, or other safety requirements, treat compliance as part of acceptance—not as an optional note after the task is marked complete. IBM Research’s ST-WEBAGENTBENCH pairs 222 web-agent tasks with safety and trustworthiness policies and evaluates six dimensions. Its Completion Under Policy metric counts a task as complete only when applicable policies are respected. For three evaluated open agents, average Completion Under Policy was less than two-thirds of nominal task completion. That result is specific to those agents and benchmark tasks; it is not a general rate for deployed agents.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Can a separate verifier be trusted?

A separate verifier can reduce reliance on an executor’s own account of its work, but it is not automatically correct. Ask what evidence it inspected, how its criteria were derived, whether it can check the relevant state, and how its judgments were tested against human assessments. A verifier that sees only tool logs may miss whether the desired result exists. One that sees only a final screenshot may miss relevant evidence earlier in a long sequence; Microsoft Research also cautions that showing too many screenshots can overwhelm a judge.

Microsoft Research’s April 21, 2026 article, “The Art of Building Verifiers for Computer Use Agents,” reports results from a benchmark built around 246 human-labeled trajectories with process and outcome annotations. In that evaluation setup, the Universal Verifier had Cohen’s κ of 0.64 agreement with human labels. The article reports false-positive rates of at least 45% for WebVoyager and at least 22% for WebJudge relative to those labels, and describes 96 experiments in the team’s verifier-design work. These are results for the article’s evaluation setup, not universal error rates for every verifier or agent.

Another proposed design separates planning, execution, and verification: a Planner turns the user’s goal into a checklist, an Executor performs the task, and a Verifier checks the Executor’s raw output against that checklist. The public repository “The Verifier Agent: Mitigating Task Verification Failures in Agentic AI” reports a 20-task experiment and manual ground-truth review of 120 experimental runs. The accessed repository page does not state a publication year. Its authors describe the work as using single-step, atomic tasks, with more complex multi-step workflows left for future work; treat its findings as preliminary rather than evidence of general performance.

Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

What the cited approaches check—and what their results mean

Approach Evidence or scoring focus Reported scope Important boundary
Microsoft Research, “The Art of Building Verifiers for Computer Use Agents” (April 21, 2026) Separate process scoring from binary outcome scoring; compare verifier judgments with human labels. 246 human-labeled trajectories; 96 verifier-design experiments. Reported κ was 0.64; false-positive rates were at least 45% for WebVoyager and at least 22% for WebJudge relative to the article’s human labels. Figures apply to the article’s evaluation setup, not all agents or verifiers.
IBM Research, ST-WEBAGENTBENCH (July 13, 2025) Task completion under applicable safety and trustworthiness policies, alongside six evaluation dimensions. 222 tasks; three open agents evaluated. Average Completion Under Policy was less than two-thirds of nominal task completion. The reported result is limited to the benchmark and agents evaluated.
Agent-Diff (February 11, 2026) Whether the expected environment-state change occurred. 224 sandboxed enterprise workflow tasks and nine language models across four named service interfaces. Sandboxed benchmark measurements are not production reliability estimates.
“The Verifier Agent” public GitHub repository (publication year not stated on the accessed page) A separate Verifier checks Executor output against a Planner-generated checklist. 20 tasks; 120 experimental runs manually reviewed against ground truth. The described study uses single-step, atomic tasks; complex multi-step workflows remain outside its demonstrated scope.

These are different designs and evaluation scopes, not a standardized head-to-head comparison. Their results are useful for understanding what a verification method checks and where its evidence comes from; they do not establish that one method is universally best.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

A practical acceptance check for an agent’s work

  1. Write down the requested outcome. Convert the user’s request into explicit conditions that can be checked independently. Do not add unstated requirements.
  2. Inspect the relevant system. Verify the resulting record, message, setting, or other intended change in its target environment. Treat logs and status messages as supporting evidence, not substitutes for the result.
  3. Check for partial completion and side effects. Record which conditions were met, which were not, and whether anything else changed.
  4. Review constraints separately. Where consent, authorization, or safety policies apply, verify compliance independently of nominal completion.
  5. Classify blockers accurately. Distinguish an agent-controlled mistake from an external obstacle, while stating plainly whether the user’s goal was achieved.
  6. Interrogate the verifier. Ask what evidence it reviewed, how it obtained its criteria, what task types its evaluation covers, and whether its judgments were compared with human assessments.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Leave a Reply

Your email address will not be published. Required fields are marked *

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

More from the Wire

  1. World desk4 min
    How to Spot an AI Voice Scam Before Sending MoneyDon’t rely on how a caller sounds. Pause, call back through a known number, and verify the emergency with another trusted person before sending money.
  2. Mountain View desk4 min
    Google’s SynthID Detector: How to Check AI-Generated Images, Video and AudioGoogle’s SynthID Detector looks for an embedded watermark in supported images, video and audio. Here is what its results do—and do not—show.
  3. Redmond desk20 min
    How to create a link to File or Folder in Windows 11Windows 11 gives you several ways to point to a file or folder without moving or duplicating it. You can create a desktop shortcut,…
Recommended PC Tool
Recommended PC Tool
Windows Errors? Fix Them Before They SpreadFree repair scan
Outdated Drivers Are Slowing You DownFree scan - exact matches

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.