Free tools Windows power users keep installed
One-click scans. No signup required.
AI coding assistants can make some software tasks faster, but the evidence does not support one productivity figure for all developers or projects. A controlled exercise found developers completed a defined task faster with Copilot; a randomized study of experienced contributors working in familiar, mature repositories found they took longer with early-2025 AI tools. The difference is a reminder that task, codebase, developer, tool and measurement all matter.
What productivity means changes the answer
“Productivity” can mean elapsed time to finish a task, whether the task is completed at all, code quality, how much suggested code is accepted, or the amount of useful software a team delivers over time. Those measures are related, but they are not interchangeable. A developer can feel more productive without shipping faster; a code suggestion can be accepted without reducing total review or debugging time.
A timed exercise to build one small service is also unlike changing a large existing codebase. Real repository work can require understanding undocumented conventions, tracing dependencies, writing or adapting tests, and reviewing changes. Findings from one kind of work should not be treated as a forecast for another.
What the studies found
| Study | Setting and method | Result | What it can show |
|---|---|---|---|
| GitHub, 2022 | 95 professional developers were randomly assigned to use Copilot or not while writing a JavaScript HTTP server. | The Copilot group averaged 1 hour 11 minutes to complete the task, versus 2 hours 41 minutes without Copilot. GitHub reported the Copilot group completed the task 55% faster; completion rates were 78% and 70%, respectively. | A positive result for this defined task. It does not establish the same speed gain for larger projects or day-to-day software delivery. |
| METR, July 2025 | 16 experienced contributors to large open-source repositories completed 246 real bugs, features and refactors in repositories they knew. Issues were randomly assigned to AI-allowed or AI-disallowed conditions; tasks averaged about two hours. Participants could choose AI tools, and use was primarily Cursor Pro with Claude 3.5 or 3.7 Sonnet. | Issues in the AI-allowed condition took 19% longer on average. | A measured slowdown in this specific setting with early-2025 tools, not proof that assistants slow all developers or tasks. |
| UK Government Digital Service, November 2024–February 2025 | A public-sector trial made 2,500 licenses available across central government organizations, with 1,900 assigned. Its mixed survey and telemetry analysis included 424 responses from users in 31 departments; 73% of respondents reported at least five years of coding experience. | 58% of respondents said they would not want to return to pre-assistant working conditions; average satisfaction was 6.6 out of 10. Telemetry showed 15.8% average acceptance of suggested Copilot code lines, and 39% of respondents reported committing code suggested by the assistant. | Evidence about reported experience and patterns of use, not a randomized estimate of faster end-to-end delivery. |
| GitHub code-quality study, published 2024 and updated 2025 | Developers with at least five years of experience were randomly assigned Copilot access or no AI; 202 valid submissions were analyzed. Developers implemented web-server API endpoints, assessed using ten unit tests and blind expert review. | GitHub reported that Copilot-assisted submissions were 53.2% more likely to pass all ten unit tests. It also reported higher functionality and improvements in readability, reliability, maintainability, conciseness and approval likelihood. | A vendor-run study of a particular task and rubric. The 53.2% is a relative likelihood reported by GitHub, not a 53.2 percentage-point increase or a general production-quality guarantee. |
| Microsoft Research, June 2025 | The publication describes randomized controlled trials at Microsoft, Accenture and an anonymous Fortune 100 company, where random subsets of developers received access to an AI assistant with code completions. | The reported study description establishes the setting and design, but does not state an outcome estimate here. | It adds evidence from workplace trials, but a numerical productivity effect cannot be inferred from the design description alone. |
Why the results differ
A contained exercise versus work in a familiar codebase
The GitHub speed study asked participants to build a JavaScript HTTP server under a controlled, timed setup. The METR study instead involved experienced contributors making changes in large repositories they already knew. Familiarity may help a developer navigate a codebase, but it also means a change must fit established behavior and conventions. In that situation, generating code is only part of the work; checking whether it belongs can take time.
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →#1 Best Overall
METR noted that realistic repository tasks and algorithmically scored benchmarks are different kinds of tests. Its result should be read as a snapshot of early-2025 tools in one setting. The researchers also discussed limits to generalizing from their participants and tasks, along with possible learning and recruitment effects.
Different users, tools and interaction patterns
Study participants vary in experience, familiarity with the repository and skill using an assistant. Tools and models change as well. METR’s finding concerns the tools participants used in early 2025, primarily Cursor Pro with Claude 3.5 or 3.7 Sonnet; it should not be presented as a timeless measurement of every assistant or as a result about all tools available in 2026.
Access alone does not mean an assistant’s suggestions are used. In the UK public-sector trial, acceptance telemetry and survey responses captured different parts of the workflow. A low share of accepted suggested lines does not by itself establish low value: a suggestion may prompt a useful approach without being accepted verbatim. Conversely, acceptance does not prove a net time saving after edits, tests and review.
Different outcome measures
Elapsed time, completion rate, test results, expert ratings, acceptance telemetry and satisfaction each answer a different question. Quality matters because faster initial code is not necessarily faster delivery if it creates more defects or maintenance work. GitHub’s quality experiment assessed a specific task with tests and expert review; it does not settle code quality across production systems.
Perceived speed and measured speed can diverge
Before the METR study, participating developers expected AI to make them 24% faster. After completing the study tasks, they still believed AI had sped them up by 20%, although the measured average showed they took 19% longer in the AI-allowed condition. This gap illustrates why confidence, satisfaction and delivery time should be reported separately rather than combined into a single productivity claim.
The UK government trial also found positive sentiment: 58% of respondents said they would not want to return to pre-assistant working conditions, and average satisfaction was 6.6 out of 10. Those responses are meaningful evidence about how participants experienced the trial, but they do not substitute for a causal measurement of output or elapsed delivery time.
Rank #4
How to judge whether an assistant helps your team
For an organization deciding whether an assistant improves its own engineering work, evaluate a representative workflow instead of relying on a headline percentage. Compare similar tasks with and without the tool, and define what counts as finished before measuring.
- Choose representative work. Include the kinds of changes the team actually makes, such as bug fixes, feature work and refactors, rather than only short greenfield exercises.
- Account for context and experience. Record repository familiarity, developer experience and prior assistant use; otherwise, differences between groups may reflect more than the tool.
- Track more than typing time. Measure elapsed time through implementation, tests and review, alongside completion, defects or rework. Code generation speed alone can miss downstream costs.
- Separate use from impact. Track active use and accepted suggestions if useful, but do not treat either as a direct measure of delivery speed or business value.
- Report the conditions. Identify the assistant and model, workflow, date, task mix and study design so results remain interpretable as tools and practices change.
The available findings support a conditional conclusion: AI coding assistants can help on some tasks and may improve particular task-level quality measures, while they can also add time in complex work where developers must integrate suggestions into a mature codebase. Whether they raise productivity for a given team depends on the work and on how productivity is measured.
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
