Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Scan for outdated or missing drivers - takes under a minute3Repair Windows errors before they cause bigger problemsMeasure agentic engineering across the whole delivery path—not by lines of generated code or agent activity alone. Count accepted, quality-qualified changes alongside review effort, rework, delivery time, operating costs, and what the team actually does with any capacity it frees.
What should you measure?
Use a task or change as the unit of analysis. Define when its work starts and when it counts as accepted and released, then track whether an agent participated, the task class and complexity, repository maturity, team experience, and the agent’s level of autonomy. Those details help distinguish a tool effect from differences in the work or the team.
As an Amazon Associate I earn from qualifying purchases.
Follow the change from planning through agent execution, human review, correction, testing, integration, deployment, and post-release outcomes. Keep early activity measures separate from delivery results: agent adoption, generated code, and completed sessions describe use; they do not establish that useful work shipped.
| Dimension | What to count | How to interpret it |
|---|---|---|
| Accepted output | Changes accepted, merged, released, and meeting agreed quality gates | Prefer production-qualified changes to generated lines, pull-request counts, or session completion. |
| Review | Reviewer active time, review-queue wait, review rounds, requested changes, and acceptance or rejection | Separate time spent reviewing from elapsed queue time; a shorter coding phase may shift work to reviewers. |
| Rework | Human corrections, agent retries, failed validation loops, integration fixes, reopened changes, rollbacks, and post-merge remediation | Set attribution rules. Rework may reflect unclear requirements or repository conditions as well as agent output. |
| Flow | Lead time, throughput, deployment frequency, blocked time, and change-failure or stability measures | Read these together: throughput can rise while stability falls, and queues can hide local speed gains. |
| Quality and risk | Defects, escaped defects, security findings, maintainability, architectural fit, and reliability | Apply the same quality gates and thresholds in comparisons. |
| Full cost | Human time, review and rework time, model and token spend, licenses, compute, sandbox and CI costs, integration, governance, and training | Tool spend alone is not the total cost of delivery. IBM identifies review, validation, rework, governance, training, infrastructure, and integration among costs that can be less visible than licenses and tokens (IBM, 2026). |
| Realized value | Product or customer outcomes, roadmap delivery, avoided cost, risk reduction, and capacity redeployed | Name the value mechanism and the evidence for it. Freed hours are potential capacity, not realized value by themselves. |
How can you tell whether the agent is saving time?
Compare like work against a baseline and retain the distribution of results—not just a team average. Include production time and review-queue time: faster generation can still mean slower delivery if reviewers, integration, or validation become bottlenecks.
#1 Best Overall
- Define the outcome and clock. Specify what qualifies as an accepted, released change and when the task clock starts and stops. Keep the definition consistent between agent-assisted and baseline work.
- Classify the work. Record task type and complexity, repository maturity, team experience, and agent autonomy. Compare similar tasks rather than pooling unrelated work.
- Capture time and rework across the path. Separate agent execution, human review, correction, testing, integration, and blocked or queued time. Record retries and post-release remediation under stated attribution rules.
- Apply unchanged quality gates. Compare defects, security findings, maintainability, architectural fit, and reliability using the same thresholds in both groups.
- Report distributions and context. Show variation and relevant task or team differences, not only an aggregate average. Treat survey associations and vendor telemetry as evidence about their stated populations, not causal estimates.
- Track where capacity went. If delivery work takes less time, record whether the released capacity went to roadmap work, platform modernization, new products, or another defined outcome.
For the review measure, keep active reviewer effort distinct from elapsed wait; for rework, make clear what is attributed to agent output and what may stem from requirements, repository conditions, or other causes. This makes it possible to see whether work moved downstream rather than disappeared.
What do published productivity results show?
Results depend on the task, population, tool, and measurement design. A scoped programming exercise, a real issue in a mature repository, a survey, and product telemetry do not measure the same thing.
| Evidence | Reported result | Scope and limitation |
|---|---|---|
| Peng, Kalliamvakou, Cihon, and Demirer, 2023, as summarized by the Montana Research Foundation | Participants completed a scoped JavaScript HTTP server task 55.8% faster with Copilot. | A controlled, scoped task; it is not a universal estimate of engineering productivity. Montana Research Foundation, 2026 synthesis. |
| METR, 2025, as summarized by IBM and the Montana Research Foundation | Experienced developers took 19% longer on real issues when allowed to use AI. | The trial involved 16 experienced open-source developers and 246 real issues, according to the Montana Research Foundation. IBM says much of the time cost came from review, correction, and integration. IBM, 2026 account; Montana Research Foundation, 2026. |
| Later METR study, as described by IBM | IBM reports that a later study using late-2025 agentic tools found overall productivity improved. | This is a distinct study and tool context from the mid-2025 trial; the IBM account does not provide a comparable effect size here. IBM, 2026. |
| DORA, 2024, as summarized by the Montana Research Foundation | A 25% increase in AI adoption was associated with 1.5% lower delivery throughput and 7.2% lower delivery stability. | These are reported associations, not proof that increased adoption caused the changes. Montana Research Foundation, 2026 synthesis. |
| McKinsey, May 2026 survey, cited in a later article | 86% of top-accelerating organizations tracked outcome metrics such as quality, productivity, and speed. | The survey included 334 respondents, with a director-level-and-above analysis of 138. The finding does not show that outcome tracking caused acceleration. McKinsey. |
| Anthropic Claude Code session analysis, October 2025–April 2026 | Anthropic estimated that typical task value rose about 25% on average over the observed period. | The analysis covered about 400,000 Claude Code sessions from about 235,000 users. Its task-value estimate used comparisons with freelance job postings; it is not a cross-product productivity benchmark. Anthropic defines success as accomplishing the user’s stated aim with verifiable evidence, such as passing tests or committed work. Anthropic, June 16, 2026. |
| Weave Q2 2026 platform telemetry | Median-organization output per engineer rose 1.8x from Q3 2025 to Q2 2026. | Weave reports telemetry from 1,470 organizations and 21,409 engineers, using its own complexity-weighted output measure. This is vendor-defined, platform-specific evidence, not an industry-standard measure. Weave. |
These findings are not contradictory estimates of one fixed effect: their tasks, participants, tools, and methods differ. Use them to understand why local measurement matters, not to predict a universal gain or loss.
Recommended Free Tools
How should you calculate ROI?
There is no source-backed, standardized formula that combines agent value, review, rework, quality, and cost into one accepted industry measure. A team can define a local measure such as cost per accepted, quality-qualified change, but it should publish the denominator, quality conditions, human-time accounting, cost categories, and observation window. Do not call a locally defined score an industry standard.
Rank #3
- Count the full cost: labor across planning, review, correction, testing, integration, and release, plus model, license, compute, CI or sandbox, governance, and training costs.
- State the value mechanism: for example, faster roadmap delivery, avoided cost, reduced risk, or capacity redirected to a new product. Identify what outcome would demonstrate that value.
- Separate potential from realized value: a reduction in hours is not a financial or customer benefit unless the capacity is productively redeployed or another measurable cost or risk falls.
- Keep quality in the calculation: a cheaper or faster change that misses agreed quality gates is not equivalent to an accepted, production-qualified change.
McKinsey’s May 28, 2026 delivery article recommends deliberate capacity allocation as part of agentic workflow redesign, alongside stronger review and supervisory skills and involvement from risk and compliance roles. Its practical implication for measurement is to track what the freed capacity actually enables, not just that time was saved. McKinsey, May 28, 2026.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What can distort an agent productivity score?
- Counting output volume as value: pull requests, lines, tokens, or sessions can rise without a matching increase in accepted and useful releases.
- Omitting the review queue: coding time may fall while reviewer effort or elapsed wait rises. McKinsey describes a shift toward validating and reviewing consequential decisions as agents generate more artifacts. McKinsey.
- Ignoring correction and integration: retries, failed validations, rejected changes, and post-merge fixes are part of the delivery cost, not noise to discard.
- Pooling unlike work: task difficulty, repository condition, developer experience, and autonomy can change the result even when the tool is unchanged.
- Trading stability for throughput: interpret deployment and speed measures alongside change failures, rollbacks, escaped defects, security, and reliability.
- Treating vendor metrics as neutral standards: telemetry can be useful, but proprietary definitions and samples limit comparison across vendors or teams.
- Assuming good tools repair weak engineering practice: SIG’s State of Software 2026 release reports findings from a benchmark spanning more than 30,000 systems and 400 billion lines of code; its current-year findings draw on systems analyzed over the prior year. Its AI-code, maintainability, architecture, and security findings reflect SIG’s methods and benchmark population. SIG argues AI can amplify sound or weak engineering discipline. SIG, State of Software 2026.
No regulator or standards body is established here as requiring a particular agentic-engineering measurement method. Teams should document their own definitions and comparison rules rather than imply that a formal ROI standard exists.
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
