The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →AI can help developers complete work while leaving them with less understanding of what they built—but that is not a universal or proven long-term effect. Studies measure different things: immediate learning, task throughput, code volume, completion time, and employees’ own perceptions. Their results point to a real tension, not a settled verdict that AI makes engineers worse or employees better.
Does AI make software engineers worse at coding?
One randomized study found lower immediate mastery among developers who used AI while learning an unfamiliar library. Anthropic’s January 2026 study involved 52 mostly junior software engineers who used Python regularly but had not worked with Trio, an asynchronous Python library. Participants completed two feature tasks in an online coding platform and then took a quiz on debugging, code reading, code writing, and concepts.
As an Amazon Associate I earn from qualifying purchases.
The AI-assisted group averaged 50% on the quiz, compared with 67% for participants who hand-coded—a 17-percentage-point gap that the study reported as statistically significant. The largest gap was on debugging questions. AI users finished about two minutes sooner on average, but that time difference was not statistically significant.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Scan for outdated or missing drivers - takes under a minute3Repair Windows errors before they cause bigger problemsThis is evidence about near-term mastery after a short learning task, not a general measure of engineering ability. The assistant was a sidebar tool, not an agentic coding product, and the quiz came shortly after the task. The study authors note: “Whether immediate quiz performance predicts longer-term skill development is an important question this study does not resolve.” It therefore does not show that AI causes lasting deskilling.
#1 Best Overall
Anthropic’s analysis of screen recordings also found patterns worth investigating: lower-scoring participants more often delegated heavily or asked AI to debug and verify their work, while higher-scoring groups more often asked conceptual questions and requested code explanations. The authors explicitly say this qualitative analysis does not establish that those interaction patterns caused the different quiz results. Treat them as hypotheses, not proven learning techniques.
Do AI coding assistants improve productivity?
Some workplace experiments found more output with AI assistance; another found experienced developers taking longer on demanding repository work. The measures and settings differ, so these results should not be averaged into a single claim about AI productivity.
| Study and setting | Reported result | What the measure means |
|---|---|---|
| Microsoft Research, June 2025: randomized field experiments at Microsoft, Accenture, and an anonymous Fortune 100 company; 4,867 developers in total. | AI-assistant access was associated with 26.08% more completed tasks in aggregate; standard error was 10.3%. Individual experiments were noisy. | Completed-task throughput in those companies. Less experienced developers had higher adoption and larger reported gains; the figure does not measure code quality, learning, or organizational value by the same percentage. |
| Bank for International Settlements, Working Paper 1208, September 2024: Ant Group’s CodeFuse field experiment, following its September 2023 introduction. | The treatment group produced 55% more lines of code, with statistically significant gains primarily among junior employees. | Lines of code, not quality-adjusted productivity. About one-third of the increase was directly attributed to generated code; the authors interpreted the rest as likely efficiency gains elsewhere. |
| METR, July 2025: randomized trial of 16 experienced developers working on their own mature open-source repositories, across 246 real issues, using early-2025 AI tools. | Developers took 19% longer to complete issues when AI was allowed. | Completion time in this specific setting. METR cautions that the result does not establish that most developers are slowed down or represent routine work, novice developers, or later tools. |
The studies answer different questions. A completed task, a line of code, and time to resolve an issue are not interchangeable outcomes; none alone establishes whether the result is maintainable, correct, or valuable to an organization. The Microsoft and BIS summaries report larger gains among less experienced or junior staff, while Anthropic’s mostly junior sample showed lower immediate quiz performance during a learning task. Together, these findings raise a training question, but they do not show that the workers in the productivity experiments lost skills.
Can AI help developers work faster while weakening their skills?
In principle, finishing a task with assistance and learning to do the task independently can move in different directions. A tool may help someone produce an answer without requiring them to practice every step; conversely, using it to explore unfamiliar code or clarify a concept may support understanding. The cited studies do not establish which balance will hold over months or years.
That distinction matters especially for early-career engineers. A manager who tracks only immediate delivery can miss whether someone is developing the judgment to debug, review, and maintain the work later. A developer who wants to learn can use AI as a source of explanations, but should still test whether they can reason through the code without leaning on the answer it supplied.
Does AI make employees feel better about work?
Microsoft Research’s 2025 “Dear Diary” study combined surveys, a randomized controlled trial, and a three-week diary study at one large multinational software company. It reported more positive perceptions of tool usefulness and enjoyment after introduction and sustained use, while perceptions of code trustworthiness did not change. In that study, 84% of participants reported positive changes in daily work, and 66% reported some change in how they felt about work.
These are participant reports from one company, not objective productivity measures or representative statistics for all workers. Diaries also captured mixed experiences: some participants welcomed the tools, while others described greater pressure to keep up with changing technology. “Better employee” is therefore too broad if it means universally happier, more capable, or more productive.
How should developers and managers use AI without confusing output with capability?
The studies do not prove that any particular workflow prevents skill loss. Still, a cautious approach follows from the difference between completing work and demonstrating understanding:
Best Value
- For learning tasks: Try to outline the approach before prompting. Ask the assistant to explain unfamiliar code or concepts, then verify the explanation and work through at least one debugging or testing step yourself. Treat this as a sensible practice, not a proven intervention.
- For code review: Keep responsibility with the engineer. Check behavior, tests, edge cases, dependencies, and maintainability rather than accepting plausible-looking generated code.
- For team evaluation: Pair delivery measures with review quality, rework, defects, and whether developers can explain and maintain what they ship. Do not use task counts or code volume as stand-ins for quality or long-term capability.
- For tool decisions: Evaluate the actual work and workforce in front of you. Results from a short unfamiliar-library exercise, company task experiments, and mature open-source repositories may not transfer to one another.
The defensible reading of the evidence is narrower than the headline: AI assistance can coincide with higher workplace output and more positive reported experiences, while one controlled learning study found lower immediate mastery and one open-source trial found slower completion. Whether that trade-off becomes lasting skill loss—or a general improvement in work—remains unsettled.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
