Developers who learned to code before AI assistants can welcome the technology and still ask whether it helps. The available evidence does not establish what any particular group of experienced developers believes; it does show why blanket claims about AI making developers faster are hard to defend. Workplace experiments found more completed tasks, a small trial in mature open-source projects found slower completion, and a vendor study reported better results on a bounded code-quality exercise. Those findings measure different things in different settings. The practical question is not whether to be “pro” or “anti” AI, but what an assistant does to the work your team actually needs to ship.
What the evidence says—and why results differ
“Productivity” can mean more tasks finished, less time per task, code that passes tests, or software that is easier to maintain. A result on one measure does not automatically establish gains on the others. The studies below differ in participants, tasks, tools, and settings, so they should be read as evidence about particular situations rather than as competing verdicts on AI coding assistants as a whole.
As an Amazon Associate I earn from qualifying purchases.
Workplace experiments counted completed tasks
Microsoft Research’s June 2025 analysis combined three randomized field experiments at Microsoft, Accenture, and an anonymous Fortune 100 company. Across 4,867 developers, the researchers reported 26.08% more completed tasks among developers offered an AI coding assistant, with a standard error of 10.3%. The reported gains were larger among less experienced developers, who also adopted the tools more. This is a task-count result, not evidence that every developer worked 26% faster or spent 26% less time on a given task. Microsoft Research’s account of the field experiments describes the study and its findings.
A trial in familiar, mature projects found a slowdown
Becker, Rush, Barnes, and Rein studied 16 experienced open-source developers completing 246 tasks in mature projects they already knew. When AI was allowed, participants primarily used Cursor Pro and Claude 3.5 or 3.7 Sonnet. In this early-2025 study setting, task completion took 19% longer with AI tools, even though participants had forecast time savings. The authors report that the slowdown was robust across their analyses, while noting that experimental artifacts cannot be entirely ruled out. The small trial does not settle how assistants perform in other workplaces or on other kinds of work; it does challenge the assumption that experience plus an assistant necessarily means faster completion. The study preprint details its design and limitations.
#1 Best Overall
A code-quality test measured one constrained exercise
GitHub recruited 243 developers with at least five years of Python experience for a study of Copilot; 202 provided valid submissions, with 104 assigned to use Copilot and 98 to work without AI. Participants wrote API endpoints for a fictional restaurant-review web server. The code was evaluated with ten unit tests and blind developer reviews. GitHub reports that the Copilot group was 53.2% more likely to pass all ten tests, produced 13.6% more lines per readability error, and had statistically significant relative improvements in readability (3.62%), reliability (2.94%), maintainability (2.47%), and conciseness (4.16%). These are results from a single exercise in a vendor-authored study—not guarantees about production code, every language, or every dimension of quality. GitHub’s study report explains its method and measures.
The studies do not test the same claim
The workplace experiments measured completed-task counts across organizations; the open-source trial measured elapsed completion time for work in projects familiar to participants; the Copilot study assessed submissions against tests and code-review measures in a defined exercise. A tool can help with some tasks or measures and hinder others. Differences in task complexity, codebase familiarity, tool versions, developer experience, and review effort matter when applying a study to a team’s work.
Adoption is not proof of productivity
A GitHub/Wakefield Research online survey fielded from February 26 to March 18, 2024, asked 2,000 non-student, non-manager enterprise respondents at companies with more than 1,000 employees across the United States, Brazil, Germany, and India. More than 97% said they had used AI coding tools at some point. That figure describes ever-use among this sample—not daily use, company-approved use, or a measured performance gain. The survey did not ask how often respondents used the tools and notes that company sanction is a separate issue. The survey summary sets out its scope and caveats.
AI’s effects depend on the organization around it
DORA’s 2025 report draws on more than 100 hours of qualitative data and survey responses from nearly 5,000 technology professionals around the world. Its authors frame AI as an amplifier of organizational strengths and dysfunctions. That is a systems-level interpretation, not a precise causal estimate of how much productivity AI creates. It points to a useful practical distinction: an assistant may generate code, but whether that code helps depends partly on the surrounding work—such as the clarity of tasks, the quality of review, and how teams handle integration and maintenance. Google Research’s page for the DORA report describes its evidence base and framing.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.How a team can test whether an assistant helps
For a team deciding whether to adopt or expand an AI coding assistant, the most relevant evidence is a comparison using its own work and quality bar. A useful evaluation makes the outcome explicit before the trial begins, so “more code” or “people like it” does not stand in for improved delivery.
- Choose representative work. Include the kinds of tasks developers actually do, such as changes in familiar codebases as well as bounded new-code work. Record task type and complexity so results are not treated as universal.
- Compare comparable conditions. Where practical, compare similar tasks with and without the assistant. Record the tool and model version, developer experience, and whether the codebase was familiar; these factors can change what a result means.
- Measure more than output volume. Track completion time and tasks completed separately. Also examine tests, defects, review effort, maintainability, and rework where those measures fit the work. A faster first draft may not be a faster finished change.
- Include the costs around generation. Measure time spent checking suggestions, correcting errors, integrating changes, and maintaining the result. If those costs are invisible, a trial can overstate the benefit.
- Review the result by task and developer group. An average may hide that the assistant helps with one type of work but slows another, or affects less- and more-experienced developers differently. Keep the conclusion within the scope of the tasks and participants tested.
This approach does not assume AI will help or harm. It makes the decision answerable: under these conditions, for these tasks, did the assistant improve the outcome the team values?
Quick Recap
Best Value
Rank #4
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Free tools Windows power users keep installed
One-click scans. No signup required.
