Free tools Windows power users keep installed
One-click scans. No signup required.
Not in any way the evidence can yet establish across software teams generally. AI coding assistants have increased task throughput in some studies and saved time according to some users, but another randomized study found experienced developers took longer on familiar projects. Most studies measure tasks, time estimates or tool activity—not the full cost of producing and maintaining reliable software.
What the evidence says—and what it measures
The answer depends on what “cheaper” means. A team might finish more tasks in a given period, spend fewer hours on a task, or reduce the total cost of delivering and maintaining software. Those are related but different outcomes. A productivity result does not become a cost saving until it is translated into useful work and compared with all the costs involved.
The studies below differ in participants, tasks, tools and methods. Their headline figures should not be treated as competing measurements of one universal AI effect.
| Evidence | Result | What it measures |
|---|---|---|
| Microsoft Research field experiments, 2025 | 26.08% more completed tasks on average; standard error 10.3% | Pooled task throughput across randomized deployments at three companies; not total cost. |
| METR randomized study, 2025 | 19% longer task completion time; 2026 update gives a 2%–39% interval for longer time | Completion time for tasks by experienced open-source developers in familiar repositories using early-2025 AI tools. |
| UK Government Digital Service trial, 2024–25 | 56 minutes saved per working day on average, as reported by survey respondents | Self-reported time savings during a three-month public-sector trial; not audited net savings. |
Why one study found more tasks completed
Microsoft Research’s June 2025 paper pooled randomized field experiments conducted during ordinary business at Microsoft, Accenture and an anonymous Fortune 100 company. A subset of developers received an AI coding assistant that suggested code completions. Across 4,867 developers, the authors estimated an average 26.08% increase in completed tasks, with a 10.3% standard error. They also reported that less experienced developers adopted the tool more and had greater productivity gains. Read the Microsoft Research study.
#1 Best Overall
This is meaningful field evidence for a positive task-throughput effect in those settings, but it does not show that overall development became 26.08% cheaper. Completed tasks are not all equally valuable, and the estimate does not by itself account for license fees, onboarding, review, defects or maintenance.
Why experienced developers sometimes took longer
METR’s July 2025 randomized study involved 16 experienced open-source developers completing 246 tasks in mature projects. Participants averaged five years of experience in the repositories they worked on. With early-2025 AI tools allowed, measured task completion time increased by 19%. Participants mainly used Cursor Pro and Claude 3.5 or 3.7 Sonnet. Read METR’s study.
The mismatch between expectation and measurement is notable: participants forecast a 24% time reduction and later estimated that AI had reduced their time by 20%, while the measured result was longer task completion. That does not establish that AI slows all developers. It does show why impressions of speed should not substitute for task-level measurement, particularly for complex work in a codebase the developer already knows well.
What METR’s 2026 update changes
In February 2026, METR said its later experiment was affected by selection effects: some developers did not want to work without AI, some tasks were withheld because participants did not want to attempt them without AI, and tracking time was difficult for some participants using multiple agents. METR described those later results as an unreliable signal of the current productivity effect. It said the early-2025 estimate’s confidence interval ranged from 2% to 39% longer task time; it also said the effect may have improved by early 2026, but the follow-up data are weak evidence for how much. Read METR’s update. The update is a reason not to project the 2025 finding uncritically onto today’s tools, not proof of a quantified current speedup.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Repair Windows errors before they cause bigger problems3Fix the driver behind crashes, sound loss and screen glitchesWhat the UK public-sector trial found
The UK Government Digital Service ran a three-month trial from November 2024 to February 2025, distributing licenses across more than 50 public-sector organizations. Its main analysis included 424 survey responses from users in 31 departments; 73% of respondents said they had at least five years of coding experience. Respondents reported saving an average of 56 minutes per working day when using AI coding assistants, with the largest reported savings in code creation and analysis.
That figure is reported time saved, not an independent audit of total cost. In separate Copilot telemetry, the average acceptance rate for suggested code lines was 15.8%; 39% of users said they had committed code suggested by the assistant. Acceptance is not the same as useful, production-ready output, and neither statistic establishes net savings after review or rework. Read the Government Digital Service trial report.
Rank #4
Why organizational conditions matter
DORA’s 2025 research drew on more than 100 hours of qualitative data and survey responses from nearly 5,000 technology professionals worldwide. Its report describes AI as an amplifier of existing organizational strengths and weaknesses, arguing that the greatest returns come from attention to the underlying organizational system rather than tools alone. Read the DORA 2025 report overview and Google Research’s report record.
For a team evaluating AI, that framing makes workflow as important as the assistant: whether work is clearly scoped, code review is effective, and the delivery process can absorb AI-generated changes without increasing defects or coordination overhead. DORA’s findings provide organizational context; they do not demonstrate a net cost reduction for every team.
Recommended Free Tools
Best Value
Why vendor figures do not settle the cost question
GitHub’s economic-impact article reports that an earlier quantitative study found developers completed tasks 55% faster with GitHub Copilot, and that users accepted nearly 30% of suggestions on average during the product’s first year. These are vendor-published figures, not direct calculations of total development cost. Read GitHub’s article on its productivity study.
The same article projects a possible boost of more than $1.5 trillion to global GDP. That scenario assumes a 30% productivity enhancement and 45 million professional developers in 2030. It is a conditional macroeconomic projection, not an observed saving or a direct estimate of the cost to build software. Read GitHub’s economic-impact analysis.
How to tell whether AI lowers your team’s costs
A meaningful comparison should hold the work and quality bar steady, then count both the useful output and the resources required to produce it. A team can run a controlled comparison on a stable set of comparable tasks, assigning AI access consistently and tracking results over a period long enough to capture follow-up work.
- Define the outcome: compare completed, accepted work—not suggestions generated or lines typed—and decide how to account for differences in task difficulty and value.
- Measure the full workflow: record implementation, prompting, supervision, code review, debugging, testing, security review and rework time.
- Include adoption costs: count tool and model fees, onboarding, training and any changes needed to integrate AI into the team’s process.
- Check quality and later costs: track defects, reversions, incidents and maintenance effort, not just the time until a task first appears complete.
- Compare like with like: separate results by developer experience, task type and codebase familiarity; short, well-defined work may behave differently from changes to a mature system.
- Judge the net result: compare the cost of producing work that meets the same quality standard over an appropriate time horizon.
This accounting is necessary because none of the cited studies establishes a representative, fully loaded net-cost reduction across software teams after fees, adoption, oversight, rework, security and maintenance are counted.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
