The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →For terminal agents, checking several candidate commands before running one can outperform rerunning complete task trajectories—but only when the verifier can identify the right command. In one Mid-Harness TerminalBench-Lite comparison, combining action-level verification with Best-of-3 trajectories achieved a higher Pass@1 score than Best-of-7 alone at lower estimated reference-priced token cost. That is evidence for a useful strategy in a tested setting, not a guarantee that action scaling always wins.
How candidate verification works
Mid-Harness studies extra inference at the boundary between an action-generating model and the execution harness. At each step, the model proposes multiple possible next actions from the same interaction history. A verifier compares those candidates, selects one, and passes it to the existing harness for execution.
![Claude AI for Beginners Bible: [5 in 1] The Ultimate Guide to Automate Your Work, Save Hours Every Week, and Use AI for Real-World Results](https://m.media-amazon.com/images/I/415+fSJacsL._SL160_.jpg)
As an Amazon Associate I earn from qualifying purchases.
The key is timing: alternatives are assessed before they change the environment. A terminal command can alter files, install packages, or otherwise shape what the agent can do next. For example, the DEV Community article illustrates how a package-install typo—pip install yaml instead of pip install pyyaml—could send a task down a worse path. That is an illustration, not a measured result from the paper.
Trajectory scaling allocates inference differently. Rather than comparing commands at one decision point, it runs or refines multiple task trajectories and compares those broader attempts. Mid-Harness keeps the action generator and harness fixed in its central comparisons to examine what changes when more computation is spent selecting actions.
#1 Best Overall
What the benchmark results show
The central TerminalBench-Lite comparison used TMAX-9B as the action generator. The reported figures show a substantial gain when a capable verifier evaluates a pool of candidate actions:
| TerminalBench-Lite configuration | Reported Pass@1 |
|---|---|
| TMAX-9B base-agent result | 50.00% |
| Same TMAX-9B generator with eight sampled actions and a GPT-5.6 Sol verifier | 68.03% |
| Verifier-distillation comparison, with the action generator unchanged | 54.76% to 57.14% |
The paper also compares action and trajectory scaling. With TMAX-9B on TerminalBench-Lite, Mid-Harness combined with Best-of-3 trajectories scored 66.33% Pass@1, compared with 59.18% for Best-of-7 alone. The combined setting had lower estimated reference-priced token cost. These cost figures are experiment-specific estimates, not a prediction of a provider bill or universal cost per task.
Results beyond this benchmark are not uniform. For example, the paper reports TMAX-9B improving from 21.72% to 27.34% Pass@1 on Terminal-Bench 2.1 under zero-shot verification, and from 46.67% to 48.00% on the SWE-bench-Verified Mini subset. The study covers additional models, benchmarks, and harnesses; gains depend on the evaluation setting and verifier variant.
Why the verifier matters more than candidate count
Sampling creates options, not better decisions by itself. The verifier has to recognize which candidate suits the task and the current environment. In the authors’ words, “With a TMAX-9B generator, more action sampling yields little benefit under weak verification, whereas a capable verifier can exploit useful alternatives from the same generator.”
Among the evaluated self-verification methods, pairwise verification performed best. Distilling responses from a stronger verifier improved a smaller verifier without changing the action generator, but did not close the full gap to frontier verification. The authors also report persistent disagreement with the stronger verifier over command semantics and execution feasibility.
How to compare action scaling with trajectory reruns
A fair comparison needs to account for more than the number of samples. Check these dimensions in the target system:
- Task success: compare the same benchmark, task set, and metric. Pass@1 and Pass@3 are not interchangeable.
- Inference cost: distinguish raw token counts, estimated reference-priced cost, and actual deployment spend. The reported cost comparison uses estimated reference pricing.
- Verifier method: identify whether candidates are judged by a stronger external model, self-verification, pairwise comparison, or a distilled verifier.
- Environment executions: action filtering can select a returned run’s next command using one environment instance, while trajectory sampling may execute multiple complete attempts.
- Transfer: name the model, benchmark, and harness. Performance can vary across all three.
These distinctions matter when deciding whether to spend inference on more candidate commands, complete reruns, or a combination. The paper’s experiment supports combining action-level selection with trajectory sampling; it does not establish one allocation as best for every agent or task.
Recommended Free Tools
What the evidence does not establish
The reported head-to-head result supports the claim that action scaling can beat trajectory reruns in a particular evaluated configuration. It does not show that action-level scaling is always cheaper, more successful, or safer in production. Nor does it establish that any harness verifier will reliably select commands in a live environment.
Best Value
The authors note that they lack gold action labels, which limits direct measurement of candidate coverage and verification correctness. Their findings are benchmark evidence about a method, not proof of universal production safety. Likewise, the cost comparison is tied to its model mix, token counts, and reference pricing; it should not be read as a general dollar-per-task saving.
When this approach is worth testing
Action-level verification is a reasonable option to evaluate when a terminal agent often has several plausible next commands and a verifier capable of distinguishing them. Measure it against trajectory reruns on the same tasks, tracking success, inference spend, and environment executions. If the verifier is weak, simply widening the action pool may add little value; if it is capable, action selection may improve the cost-success trade-off or complement trajectory scaling.
Primary study: Mid-Harness: Scaling Actions Between Model and Harness for Terminal Agents, posted September 30, 2026. Practical illustration: Reid Marlow’s DEV Community article, posted October 1, 2026.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
