Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →Both findings can be right because they measured different work. A 2023 GitHub Copilot experiment found developers completed a tightly defined JavaScript task 55.8% faster with Copilot; a 2025 METR trial found experienced open-source developers took 19% longer with early-2025 AI tools on issues in repositories they already knew. Neither result is a universal productivity score for AI coding.
What the 55.8% faster result measured
In a 2023 study, Sida Peng, Eirini Kalliamvakou, Peter Cihon, and Mert Demirer asked recruited developers to implement a JavaScript HTTP server as quickly as possible. The group with GitHub Copilot available completed that bounded task 55.8% faster than the control group. The paper was submitted on 13 February 2023. Read the study.
As an Amazon Associate I earn from qualifying purchases.
This is evidence about elapsed time on a specific, time-pressured implementation task. It does not measure an entire team’s throughput, maintenance burden, software quality across a project, or total value delivered.
Free tools Windows power users keep installed
One-click scans. No signup required.
What METR’s 19% slower result measured
METR’s July 2025 randomized trial involved 16 experienced open-source developers working on issues in repositories they already knew. For issues randomly assigned to the condition where early-2025 AI tools were allowed, completion took 19% longer than for issues assigned to the no-AI condition. Read METR’s report.
#1 Best Overall
That result applies to those developers, tasks, tools, and study conditions. METR explicitly said the study did not show that AI fails to speed up many or most software developers, and did not claim that its participants or repositories represented most software work. METR’s report states: “We do not provide evidence that AI systems do not currently speed up many or most software developers.”
Why the results do not contradict each other
The experiments differ in the work, participants, and setting. One measured a short, self-contained server implementation under time pressure; the other measured real issues in mature codebases familiar to experienced contributors. A bounded task can make code generation a substantial share of the work. Repository issues can require understanding existing code, deciding how to approach the change, checking suggestions, and correcting them. Those workflow differences are plausible reasons the balance of time could change; the studies did not isolate them as separate causal effects.
| Dimension | GitHub Copilot study | METR study |
|---|---|---|
| Publication and tools | 2023 paper; GitHub Copilot available to the treatment group | July 2025 report on early-2025 AI tools |
| Participants | Recruited software developers | 16 experienced open-source developers |
| Work | Implement a JavaScript HTTP server under time pressure | Issues in participants’ own open-source repositories |
| Reported result | 55.8% faster task completion with Copilot | 19% longer completion time when AI tools were allowed |
| Key boundary | A timed, bounded task is not a team-wide productivity measure | METR says the sample and setting are not representative of most software work |
What later METR evidence adds—and does not establish
METR’s 24 February 2026 update describes a later experiment that began in August 2025 and involved 57 developers, 143 repositories, and more than 800 tasks. METR said the data were an unreliable signal: selection effects meant that developers less willing to work without AI, and tasks people preferred to do with AI, were less likely to appear. Reduced pay and measurement difficulties also affected the study. Its raw estimates suggested possible speedups, but METR warned that selection made them a bad proxy for real productivity. They are not a clean replication or a settled current benchmark. Read the update.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Scan for outdated or missing drivers - takes under a minute3Repair Windows errors before they cause bigger problemsIn May 2026, METR made a further distinction between “speed uplift” on a particular set of tasks and “value uplift” when AI changes which tasks people choose to do. Faster completion on a fixed task set need not mean greater overall value if the available tools change the mix of work. Read METR’s analysis.
Rank #3
METR also reported an early-2026 survey of technical workers’ self-reported AI impact. Those responses describe perceptions, not a randomized estimate of causal productivity effects; the survey was convenience-sampled. Read the survey report.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.How to judge an AI productivity claim
Before applying either percentage to a team or project, check what the study counted as productivity. Useful questions include:
- Were tasks fixed or chosen? A fixed task set measures performance on those tasks; choices altered by AI can change the broader task mix.
- How much context did the work require? A short implementation and a change inside a large, familiar codebase place different demands on exploration and review.
- What was measured? Elapsed completion time is not automatically equivalent to output quality, total completed work, or long-term value.
- Who and which tools were studied? Results for a particular participant group, assistant, and period should not be generalized to every developer or today’s tools without supporting evidence.
- Was the evidence randomized or self-reported? Perceived gains can differ from measured outcomes and may be affected by who chooses to participate.
METR’s 2025 participants expected a 24% speedup and later perceived a 20% speedup, even though the trial measured 19% longer completion time with AI allowed. Those figures describe expectations and retrospective beliefs in that study, not alternative measured productivity effects. METR’s report.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
