Free tools Windows power users keep installed
One-click scans. No signup required.
Some links on this page are affiliate links: if you buy through them we may earn a commission, at no extra cost to you.
Evaluate a browser agent as an experiment, not a single percentage. Define a checkable end state for every task, choose an environment that matches your deployment, repeat runs under documented conditions, and report success together with reliability, efficiency, trajectory diagnostics and safety. A result is interpretable only when the task set, evaluator, agent configuration, benchmark version and run date are known.
The procedure below gives you a reproducible design for comparing agents without treating scores from different benchmarks as interchangeable.
1. Define exactly what counts as success
Start with the user goal, then write the observable state that proves it was achieved. “Find a product” is not a sufficient criterion; “the product with the specified attributes is placed in the cart and the cart contains exactly one item” is testable. Prefer an environment-state check or a verified end state over a human impression.
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Choose the unit of evaluation
- Task: one complete user request, including all steps needed to reach the end state.
- Attempt: one run from a clean, documented initial state. State whether retries are allowed.
- Success condition: a deterministic assertion where possible (record created, field value changed, message sent, or page state reached).
- Failure taxonomy: record whether the agent stopped, timed out, used an invalid action, reached the wrong state, or was blocked by the environment.
WebArena was designed around functional correctness for diverse, long-horizon tasks. Its paper is a useful model for writing end conditions before running an agent: Zhou et al., WebArena (2023).
#1 Best Overall
Specify the evaluator and denominator
Document the assertion, evaluator version and denominator in the report. If a human or model judge is needed, publish the judging rubric, examples of borderline cases and how disagreements are resolved. Report attempted tasks as well as successful tasks; silently dropping timeouts or invalid runs inflates the result.
2. Match the benchmark to the deployment question
No benchmark demonstrates universal browser competence. Select environments for the workflows and web conditions you intend to support, and record the benchmark, website and task-set versions plus the run date.
Controlled, reproducible websites: WebArena
WebArena supplies functional self-hosted sites spanning e-commerce, forums, collaborative software development and content management. It is appropriate when you need repeatable, long-horizon workflows without live-site drift. The original paper reported 14.41% end-to-end success for its best GPT-4-based agent and 78.24% for humans in that 2023 study; those are historical study results, not current leaderboard claims.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →Enterprise knowledge work: WorkArena
WorkArena is a remote-hosted suite of 33 ServiceNow tasks focused on common knowledge-work activities. Use it when your question concerns enterprise forms, records and workflows rather than consumer websites. Details and task definitions are in Drouin et al. (ICML 2024).
Live public websites: WebVoyager
WebVoyager tests online sites. OpenAI’s evaluation description names examples including Amazon, GitHub and Google Maps and contrasts this live setting with WebArena’s offline self-hosted sites. Live pages, access controls and content can change, so preserve the date, URLs or task snapshots and any account or regional conditions.
Cross-benchmark research infrastructure: BrowserGym and AgentLab
BrowserGym and AgentLab aim to provide shared interfaces and experiment workflows across web benchmarks. BrowserGym’s authors identify fragmented benchmark implementations as a barrier to reliable comparison; a common interface helps, but your paper or internal report still needs the full configuration. See The BrowserGym Ecosystem for Web Agent Research.
3. Make every run reproducible
Record the following before you launch the first trial:
- Agent identity: model name and version, agent code revision, system and task prompts, tool definitions and decoding parameters.
- Browser interface: browser and version, viewport, device emulation, action API (DOM, accessibility tree, screenshots or combinations), observation resolution and frequency.
- Environment: benchmark and task-set version, website build or live-site URLs, locale, timezone, accounts, seeded data and network policy.
- Reset procedure: how cookies, storage, databases, accounts and task state are returned to baseline between attempts.
- Limits: maximum actions, wall-clock timeout, token budget, concurrent sessions and whether the agent may retry an action.
- Failure handling: policy for transient server errors, CAPTCHA or bot checks, missing elements, navigation failures and human intervention. Never repair a run silently.
- Evaluator: assertion code or judge model, version, rubric and adjudication process.
- Sampling: number of tasks, run count per task, random seeds and task order.
- Provenance: start and end timestamps, hardware or cloud region, software lockfile and any external service limits.
BrowserGym’s motivation is directly relevant: inconsistent benchmark-specific code and undocumented methods make published comparisons difficult to reproduce. A shared interface reduces friction, but it does not replace disclosure of these fields.
4. Report a metric set, not one headline number
Use a table of components so readers can see what an agent can do, how consistently it does it and what each success costs.
| Metric | Definition to publish | Useful breakdown |
|---|---|---|
| Task success | Successful attempts divided by all attempted attempts, with the assertion and denominator | Per task, category and difficulty |
| Reliability | Consistency across repeated runs and disclosed transient-failure conditions | Variance by task and failure type |
| Efficiency | Wall-clock latency, actions, tokens, resource use and disclosed cost accounting | Median, p95 and cost per successful task |
| Trajectory diagnostics | Stored action and observation traces for investigating detours and error patterns | Unnecessary actions, recovery attempts, dead ends |
| Safety and policy | Separate outcomes for prohibited actions, consent and data-handling rules | Violation severity and adjudication |
Task success
Publish the aggregate and task-level outcomes. A high average can conceal a category in which the agent consistently fails, so include a per-task or per-category table whenever feasible. WABER describes the limitation succinctly: “Most existing benchmarks evaluate agents mainly by their success rate, the percentage of tasks they complete correctly.”
Reliability under realistic failure
Repeat each task enough to expose run-to-run variance, then run a separately labeled condition with transient delays, server errors or unexpected pop-ups. WABER proposes measuring this kind of unreliability on existing benchmarks. Do not mix injected failures with the clean baseline; report the condition, injection rate and recovery policy.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Efficiency and cost
Record elapsed time from task start to verified end state, token usage, action count and relevant resource consumption. Report median and tail latency (for example, p95), not only an average. If you can account for model and browser costs, show cost per attempt and cost per successful task; state what is excluded.
Trajectory and quality diagnostics
Keep traces that let another reader inspect unnecessary navigation, repeated clicks, premature termination and recovery after errors. There is no single canonical trajectory metric established by the sources here, so name your formula if you add one and publish enough trace data to reproduce it.
Safety and policy compliance
Define prohibited actions, consent requirements, data-access boundaries and escalation rules before testing. Keep policy outcomes separate from task completion: an agent can finish a task while violating a rule, or refuse safely and therefore score zero on task success. The available literature does not establish one universal browser-agent safety score, so state your policy and evaluator explicitly.
5. A practical evaluation workflow
- Write the task card. Include user goal, initial state, allowed accounts, success assertion, forbidden actions, timeout and retry policy.
- Freeze versions. Pin the agent, model, browser, benchmark, task set, evaluator and environment image. Save a lockfile or container digest.
- Calibrate the evaluator. Run known positive and negative fixtures. If a judge model is used, measure agreement on a labeled sample before production runs.
- Run a clean baseline. Reset state before every attempt. Capture timestamps, actions, observations, token counts, errors and final assertion output.
- Repeat tasks. Use a predeclared run count and seed policy. Keep task order fixed or randomize it deliberately and report which.
- Run failure conditions. Add one transient-failure variable at a time (delay, server error or pop-up) and label the condition in your data.
- Aggregate with confidence information. Publish counts, rates and uncertainty intervals or a clear resampling method. Do not round a small sample into false precision.
- Inspect traces. Sample successes and every failure category. Verify that the evaluator, not a logging bug, produced the result.
- Archive artifacts. Store task cards, configuration, environment versions, raw logs, screenshots or videos where permitted, and the exact analysis script.
Minimal aggregation script
Save a CSV with columns task_id,success,seconds,tokens,cost, one row per attempt. This dependency-free Python script prints overall success, median latency, p95 latency, average tokens and cost per successful task.
import csv, math, statistics, sys
rows = list(csv.DictReader(open(sys.argv[1], newline='')))
success = [r for r in rows if r['success'].strip().lower() in {'1', 'true', 'yes'}]
seconds = [float(r['seconds']) for r in rows]
tokens = [float(r['tokens']) for r in rows]
costs = [float(r['cost']) for r in success if r['cost']]
def percentile(values, p):
values = sorted(values)
if not values: return float('nan')
k = (len(values) - 1) * p
lo, hi = math.floor(k), math.ceil(k)
if lo == hi: return values[lo]
return values[lo] + (values[hi] - values[lo]) * (k - lo)
print(f'attempts={len(rows)}')
print(f'success_rate={len(success)/len(rows):.4f}')
print(f'median_seconds={statistics.median(seconds):.2f}')
print(f'p95_seconds={percentile(seconds, .95):.2f}')
print(f'average_tokens={statistics.mean(tokens):.0f}')
if costs: print(f'cost_per_success=${sum(costs)/len(costs):.4f}')
This script is an analysis aid, not a benchmark standard. Add confidence intervals, per-task slices and reliability statistics appropriate to your sample before making a release decision.
6. Compare agents without overstating the evidence
Make benchmark-specific comparisons first. Match benchmark version, task list, evaluator, attempt budget, tool access, model version and date as closely as possible. If any axis differs, label the comparison as contextual rather than controlled.
OpenAI’s 2025 Computer-Using Agent evaluation page reports 58.1% on WebArena and 87.0% on WebVoyager for its experiment. The same page notes that WebVoyager tasks are mostly simpler while complex WebArena work remains difficult. These are vendor-reported, dated results from particular setups, not timeless rankings. They cannot be placed on one universal scale with WebArena’s 2023 paper figures because the agents, tasks, evaluators and experiments differ.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.“The results demonstrate that solving complex tasks is challenging: our best GPT-4-based agent only achieves an end-to-end task success rate of 14.41%, significantly lower than the human performance of 78.24%.” — Zhou et al., WebArena: A Realistic Web Environment for Building Autonomous Agents (2023)
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchSpecial offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
7. Troubleshoot misleading or unstable results
Success rate changes between runs
Check reset isolation, random seeds, account state, live-page drift and transient server conditions. Separate clean and injected-failure runs, and report the number of attempts per task.
Many timeouts or blank pages
Inspect network logs and page-load timing before blaming the model. Record whether the environment or an external site failed, then apply the same timeout and retry policy to every agent.
Evaluator says “success” when the task is wrong
Test the assertion against negative fixtures and inspect final state directly. Version the evaluator alongside the agent; changing an assertion mid-study invalidates a direct comparison.
Agents appear efficient because they stop early
Pair latency and token metrics with verified success. Report cost per successful task, not cost per attempt alone, and keep failed traces in the denominator.
Recommended Free Tools
Safety behavior is hidden by the aggregate
Publish a separate policy-compliance table with the rule, observed action, severity and adjudication. A single success score cannot show whether an agent crossed a data or consent boundary.
Best Value
Or skip the browser setup
When your evaluation needs repeatable page evidence, ScreenshotNeo provides a website screenshot API and MCP server. It accepts consent banners as a visitor and removes more than 60 known consent platforms, newsletter popups and chat widgets before capture; each cleanup step can be disabled. Bot checks or CAPTCHAs, blank pages, timeouts, failed loads and cache hits are not billed, and response headers identify the page verdict and whether the shot was billed. Its MCP tools—take_screenshot, get_page_info and capture_pdf—let Claude, Cursor or another MCP client collect evidence directly.
One GET request returns PNG, JPEG, WebP or PDF. See the parameter reference in the ScreenshotNeo documentation.
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webpimport requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);ScreenshotNeo includes full-page and element capture, 12 device presets plus custom viewports, retina scale, dark mode, PDF controls, custom CSS and JavaScript, click and wait conditions, request blocking, headers, cookies, user agents, timezone and geolocation, transparent backgrounds, resizing, selectable cache TTLs, signed links, asynchronous webhooks, bulk capture of up to 100 URLs per call, usage reporting and an OpenAPI specification. Existing parameter names used by other screenshot APIs also work for easier migration.
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Clear out junk files and repair common Windows errors3Scan for outdated or missing drivers - takes under a minuteThe Free plan includes 1,000 shots per month with no card. Paid plans start at $5 for 3,000 shots; every feature is on every plan. Create a free ScreenshotNeo account and start with the 1,000 no-card shots.
8. Publish a report readers can audit
- State the deployment question and why the selected benchmark represents it.
- Give task cards, success assertions, denominator and per-category outcomes.
- Name agent, model, prompts, browser interface, environment and evaluator versions.
- Report run count, reset method, date, seeds, limits, retries and human intervention.
- Show success, reliability, latency, resource use, cost accounting, trace diagnostics and safety outcomes separately.
- Identify live-site conditions, benchmark drift and every mismatch that prevents a controlled comparison.
- Publish raw aggregates and analysis code or explain why protected data prevents release.
Frequently Asked Questions
How many repetitions are enough for a browser-agent evaluation?
There is no universal count. Predeclare a run count that can reveal the variance you care about, then publish the count per task and an uncertainty method; a small sample should not be reported with false precision.
Should live websites and self-hosted benchmarks appear in one score?
No. Keep their results in separate sections because site volatility, access conditions, task distributions and evaluators differ. A combined score would require an explicit, published weighting and still would not create a controlled comparison.
What should I preserve when privacy rules limit screenshots or traces?
Keep the task card, configuration, evaluator outputs, timestamps, hashes and redacted event logs. Explain exactly which evidence was removed and provide synthetic or masked fixtures for independent checks where possible.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
