AI coding assistants can make a developer finish a narrowly defined task faster, yet still fail to improve—and sometimes worsen—the team’s delivery rate. The apparent contradiction comes from different tasks, codebases, tools, participants and definitions of productivity. A controlled Copilot experiment found a large speedup on one JavaScript server exercise, while a later randomized METR trial found experienced maintainers took longer when early-2025 AI tools were available in repositories they already knew.
Neither result is a universal forecast. Individual typing speed, pull-request volume or merge activity is only one part of software delivery. To learn whether AI helps your team, measure completed, reliable value from planning through production, including review, rework, coordination and developer well-being.
As an Amazon Associate I earn from qualifying purchases.
Why the results look contradictory
“Does AI make programmers faster or slower?” has no single answer because “programming speed” can refer to different things. A timed coding exercise measures elapsed time to a specified solution. A mature open-source task may include understanding unfamiliar requirements, navigating a large codebase, testing, debugging and conforming to project conventions. A team metric such as pull requests measures activity, not necessarily useful software shipped.
The studies below are therefore complementary evidence, not percentages to average. Their differences in realism, codebase familiarity, participant experience, tool generation and outcome explain why one can report faster work while another reports slower completion.
#1 Best Overall
What the major studies actually found
| Study and design | Participants and setting | AI condition | Measured outcome | Reported result | What it does not establish |
|---|---|---|---|---|---|
| METR randomized trial (2025) | 16 experienced open-source developers, completing 246 tasks in mature projects where each had about five years of experience; tasks were randomly assigned to allow or disallow AI. | Tools available from February through June 2025, primarily Cursor Pro with Claude 3.5 or 3.7 Sonnet. | Time to complete real repository tasks. | AI access increased completion time by 19%. Before working, participants forecast a 24% reduction; afterward they estimated a 20% reduction. | A universal claim that AI is useless, or that every developer and project will slow down. The authors note the sample and setting are specific and that experimental artifacts cannot be ruled out entirely. |
| Microsoft Research controlled experiment (2023) | Developers assigned to a bounded JavaScript HTTP-server implementation task. | GitHub Copilot versus a control condition. | Task completion time and whether the specified implementation was completed. | The Copilot group completed the task 55.8% faster than the control group. | How the assistant performs on long-lived codebases, team coordination, production reliability or other languages and tasks. |
| GitHub 95-developer experiment (2022; report updated 2024) | 95 developers performing the same type of controlled programming task. | GitHub Copilot compared with a control group. | Elapsed time to finish the assigned exercise. | GitHub reported average times of 1 hour 11 minutes with Copilot versus 2 hours 41 minutes without it, described as 55% faster. | An independent, cross-industry estimate. It is a vendor-authored report about a bounded exercise, not an end-to-end delivery study. |
| GitHub–Accenture enterprise study (2024) | Enterprise telemetry, randomized assignment, adoption analysis and user surveys across organizations. | Teams using GitHub Copilot, compared through organizational activity and survey measures. | Pull-request activity and merge behavior, among other measures. | 8.69% more pull requests per developer and a 15% higher pull-request merge rate were reported. | Proof of faster end-to-end delivery, higher product value, lower lead time, better reliability or lower coordination cost. Those outcomes require separate measurement. |
“Surprisingly, we find that allowing AI actually increases completion time by 19%—AI tooling slowed developers down.” — Joel Becker, Nate Rush, Beth Barnes and David Rein, METR
How task context changes the effect
Bounded exercises reward generation speed
A small assignment with a clear interface and a known definition of done lets an assistant provide boilerplate, API syntax and test scaffolding immediately. There is little architectural history to learn and little coordination with other people. In that setting, elapsed coding time is a reasonable primary outcome, so the Copilot experiments can show a substantial advantage.
Familiar repositories make hidden work visible
Maintainers working in established projects are not starting from a blank file. They must preserve conventions, understand interactions, inspect tests, assess edge cases and decide whether generated code belongs in a system they expect to support for years. An assistant may save keystrokes while adding time for prompting, waiting, reading suggestions, correcting errors and validating behavior. That mechanism is a plausible interpretation of the METR result, not a separately proven cause.
Rank #2
Experience and tool generation matter
The METR participants were experienced maintainers and used tools available in early 2025. The older Copilot experiment studied a different participant pool, task and product generation. Improvements in models or interfaces do not retroactively change either experiment, and neither study is a head-to-head ranking of current tools.
The outcome determines the headline
“Faster” can mean less time to type a solution, more accepted pull requests, a higher merge rate, shorter lead time, fewer defects or more valuable releases. These measures can move in opposite directions. More pull requests may represent useful throughput, smaller changes, or simply more generated code awaiting review.
Why a developer speedup may not become a team gain
Review becomes the bottleneck
If generation outpaces review capacity, work accumulates in pull-request queues. Reviewers must understand code they did not write, check assumptions and request changes. The author’s local time improves while the team’s cycle time stays flat or grows.
Rank #3
Coordination costs scale with change volume
More simultaneous edits increase merge conflicts, dependency questions and handoffs. A team can produce more artifacts while spending more time deciding which artifacts should ship and in what order.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minutePC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Rework hides behind activity metrics
Generated code that passes a narrow test may still need redesign, performance tuning, security review or follow-up fixes. Counting opened or merged pull requests before those costs are known confuses movement with completed value.
Quality and reliability are delayed outcomes
A productivity intervention can look successful during a sprint and fail later through incidents, rollbacks, support work or maintenance burden. Measure defects, change-failure signals and operational outcomes over a time horizon long enough to include post-release work.
Well-being affects sustainable capacity
GitHub’s discussion of the SPACE framework treats productivity as multidimensional: satisfaction and well-being; performance; activity; communication and collaboration; and efficiency and flow. A team that ships more changes but experiences poorer focus, trust or ownership has not necessarily improved its productivity.
How to measure AI coding productivity without fooling yourself
- Define the decision. Decide whether you are evaluating faster implementation, shorter lead time, better quality, improved developer experience or a combination. Write the primary outcome before collecting data.
- Describe the work. Segment tasks by size, language, risk, novelty and repository familiarity. Do not combine a greenfield exercise with a high-risk change to a mature service and call the average “AI productivity.”
- Use a comparison group. Where feasible, randomize eligible tasks or developers to AI-allowed and control conditions. If randomization is impossible, record baseline performance and adjust for task mix, experience and seasonality.
- Track the whole flow. Capture time from task start through review, merge, deployment and stabilization. Include waiting, rework, rejected suggestions, review rounds and interruptions—not just editor time.
- Pair speed with quality. Monitor escaped defects, rollback or incident indicators, test coverage appropriate to the project, review findings and follow-up maintenance. A shorter implementation that creates downstream work is not a net gain.
- Measure team interaction. Examine review queue age, handoff count, blocked time, conflict resolution and coordination load. These reveal whether faster individual work is creating a shared bottleneck.
- Include human factors. Survey satisfaction, confidence, cognitive load and willingness to keep using the tool. SPACE treats these as productivity dimensions, not optional sentiment data.
- Set a time horizon. Recheck results after onboarding and after several release cycles. Early novelty, learning costs or temporary enthusiasm can distort a short pilot.
| Layer | Useful measures | Interpretation question |
|---|---|---|
| Individual execution | Time in active implementation, completion rate, retries and assistance accepted. | Did the person finish the assigned work more efficiently? |
| Flow | Review waiting time, cycle time, merge queue age and blocked hours. | Did work move through the system faster, or merely reach the next queue sooner? |
| Quality | Review defects, post-merge fixes, incidents, rollbacks and maintenance follow-up. | Did speed create costs after the merge? |
| Team health | Collaboration friction, satisfaction, focus and perceived ownership. | Is the improvement sustainable for the people doing the work? |
| Business delivery | Lead time to a usable release, reliability and outcome-specific product measures. | Did customers receive dependable value sooner? |
A practical team pilot
1. Choose a narrow, representative scope
Select one or two repositories and a task class that occurs regularly. Exclude emergency work and unusually risky changes from the first comparison, or analyze them separately. Record language, service area, task size and whether the developer already knows the code.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
2. Establish a baseline
Use recent historical work to estimate normal cycle time, review delay, defect rate and developer experience. A baseline prevents a busy release period or an unusually easy sprint from being mistaken for an AI effect.
3. Randomize or match
Randomly assign comparable tasks to AI-allowed and control conditions when practical. Otherwise, match tasks by complexity and compare developers with similar experience. Document who had access to which model and version.
4. Instrument unobtrusively
Use existing issue, version-control, review and deployment timestamps, then supplement them with a brief task log. Ask developers to note substantial prompting, waiting, discarded output and rework; do not require intrusive keystroke surveillance.
5. Review safety and privacy
Define what code or data may be sent to an assistant, how suggestions are logged and who can inspect pilot data. Keep generated code subject to the same tests, review standards and security controls as handwritten code.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →6. Analyze distributions, not just averages
Report medians and ranges by task category. A few very large changes can dominate an average. Examine whether effects differ for unfamiliar code, routine maintenance, tests, documentation and high-risk components.
7. Decide with a written rule
Continue, modify or stop the pilot based on pre-agreed thresholds for delivery, quality and team health. Do not expand solely because activity rose, and do not abandon a tool solely because one task class showed a slowdown.
Common interpretation mistakes
- Mixing unlike studies: A bounded JavaScript exercise and a maintainer’s repository task answer different questions.
- Treating a vendor report as a universal benchmark: The GitHub experiments are informative but specific to their designs and measures.
- Calling correlation causation: Pull-request and merge changes can reflect adoption, workflow changes or task mix rather than AI alone.
- Ignoring tool and model date: Results from early-2025 systems cannot be assumed to describe every later release.
- Optimizing one metric: Increasing activity while degrading quality, flow or well-being is not a successful productivity strategy.
- Assuming self-predictions are evidence: In the METR trial, developers expected a speedup and later still believed they had been faster, despite measured completion taking longer. Perception is useful feedback, not a substitute for outcome data.
What developers and engineering leaders should do now
For developers
- Use assistance where the acceptance criteria and tests are clear, and switch to deliberate manual reasoning when architecture, security or unfamiliar dependencies dominate.
- Review every generated change as if you inherited it. Check assumptions, failure paths, performance and project conventions before requesting review.
- Track your own cycle time and rework by task type. This shows where the tool helps you rather than where it merely produces more text.
For engineering leaders
- Describe the goal as a delivery outcome, not “more AI usage.”
- Protect reviewer capacity and monitor queue age during adoption.
- Publish the model, product version, eligible tasks and measurement window so results remain interpretable.
- Combine telemetry with developer interviews and surveys; neither alone captures the full SPACE picture.
- Keep a control or baseline long enough to detect delayed quality and maintenance costs.
The bottom line
AI can make a programmer faster on a constrained task while making a maintainer slower in a familiar, complex repository. The evidence does not establish a universal causal rule that AI makes teams slower, nor does it show that individual speedups automatically improve delivery. Treat task time, pull-request activity and merge rate as separate signals, then judge the technology by reliable work that clears review, reaches users and remains maintainable.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
