Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Measure adoption and impact as separate questions. Usage telemetry can tell you who has access to AI coding tools, who uses them, and which features they use. It cannot, by itself, show whether engineering work is faster, better, or more valuable. Pair it with a small set of delivery, quality, operational, and developer-experience measures, then compare results against a defined baseline using a method that accounts for differences in teams and work.
The question “Is AI actually helping the team?” is best answered with evidence from both systems and people: what changed, for whom, on what work, and at what cost.
As an Amazon Associate I earn from qualifying purchases.
Decide what you are measuring before collecting numbers
Start with a specific decision or question: whether to expand access, improve onboarding, change a workflow, or determine whether a tool is useful for a particular kind of work. Then define the unit of analysis. An individual task, a team workflow, a business unit, and an entire organization are different measurement problems; an aggregate result may conceal important differences among them.
Write down the scope and definitions before establishing a baseline. Specify which tools and features count, what qualifies as adoption, which outcomes matter, and the baseline and follow-up periods. If teams can use multiple tools, record tool and feature exposure rather than treating all “AI use” as one uniform intervention. Keep definitions consistent across comparison periods.
#1 Best Overall
- Reach: How many eligible engineers can use the tools, and how many actually do?
- Depth: Which features are used, how often, and in what workflows?
- Impact: Did relevant engineering outcomes change, and did quality, reliability, or developer experience change with them?
Measure adoption without mistaking activity for value
Adoption metrics are useful leading indicators. They can reveal whether people have access, whether onboarding is working, and where workflow or feature friction may exist. They are not a productivity score.
Track access and active use
Begin with license allocation as a share of purchased licenses, then count unique daily, weekly, and monthly active users. State the denominator and time window for each measure. For example, “weekly active users” is interpretable only when readers know whether it means users with access, all eligible engineers, or another defined group.
Look at movement between adoption cohorts over time as well as the overall total. This helps distinguish initial activation from continued use, while avoiding the assumption that every engineer should use a tool equally often. Low use can indicate an access, training, or workflow-fit issue; it does not, on its own, establish poor performance.
Track feature engagement
Where the tool reports them, add feature-level measures such as suggestions shown and accepted, chat interactions, agent use, and usage by language or mode. Interpret each in the context of the tool’s reporting scope. GitHub’s Copilot usage documentation distinguishes daily and weekly active users, active licensed users, suggestion acceptance, feature engagement, adoption cohorts, and an adoption multiplier that relates engaged and passive users to pull requests merged per user and time to merge. GitHub also notes that its dashboard charts do not include Copilot CLI usage.
Do not use acceptance rate or accepted lines of code as a stand-alone measure of engineer effectiveness. A higher count can reflect more suggestions, a different task mix, or different feature behavior—not necessarily better outcomes. DORA’s 2025 report lists allocated licenses, daily active users, suggestions generated, chat exposures, suggestions accepted, and lines accepted as possible early adoption signals, and cautions that these metrics alone do not assess impact.
Construct team views carefully
Individual-level usage data does not automatically make a valid team metric. GitHub says its user-team report must be joined with per-user usage metrics to build team-level measures; those metrics are not pre-aggregated in the report. Preserve the mapping and denominator used in the join, and check that team membership and reporting periods align. Otherwise, missing users or mismatched periods can distort team comparisons.
Choose outcomes that reflect engineering value
Use a small outcome set tied to the benefit you expect, rather than a single proxy such as lines of code or pull-request count. Keep measures that could reveal costs or trade-offs alongside any hoped-for gains.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Scan for outdated or missing drivers - takes under a minute3Repair Windows errors before they cause bigger problems| Dimension | Possible measures | Interpretation to protect |
|---|---|---|
| Delivery | Completed and merged work, throughput, time to merge, and end-to-end lead or cycle time | Use consistent work definitions and time windows; more activity is not automatically more value. |
| Quality | Review rework, defects, escaped defects, maintainability, and test outcomes your organization already measures reliably | Watch for extra review or correction work that could offset faster drafting. |
| Operational performance | Service reliability, change-related incidents, recovery time, and deployment outcomes | Pair delivery changes with stability measures rather than treating speed as the only result. |
| Developer experience | Perceived usefulness, cognitive load, satisfaction, flow, and time available for valuable versus repetitive work | Use practitioner feedback to explain how work changed, not as a substitute for system outcomes. |
| Business or mission outcomes | Customer outcomes or mission measures, where a plausible connection to engineering work can be established | Do not attribute a broad business change to a coding tool without a credible connection and comparison. |
Pick the dimensions that match the intended benefit and can be measured reliably. GitHub’s impact dashboard connects adoption cohorts with pull-request output and merge time. Those are useful signals to investigate, but an observational association between adoption and output is not proof that the tool caused a change.
Compare results with a credible baseline
Record the baseline before rollout: definitions, time period, eligible population, tool exposure, and the outcome measures you will examine. When practical, randomly assign access or use a staged rollout that creates a comparable group not yet exposed. A randomized comparison can support a stronger causal claim than a simple before-and-after chart, though its findings still apply most directly to the population, tasks, tools, and period studied.
If random assignment is not feasible, compare similar teams or tasks over time and document factors that could change results independently of AI use. Relevant context may include staffing, project mix, release policy, incidents, seasonality, or parallel process improvements. Report the sample size, observation period, exposure, and uncertainty; do not imply greater certainty than the design supports.
Surveys and self-reported time savings can reveal whether people feel helped and where they think time went. Treat those reports as evidence about experience, not as a stand-alone measurement of productivity. Combine them with telemetry and outcome data, and explain what each method can and cannot establish.
What published studies do—and do not—show
GitHub and Accenture’s 2024 Copilot study combined DevOps telemetry and participant surveys. It included a randomized controlled trial as well as a separate company-wide adoption analysis. In that study, 67% of participants reported using Copilot at least five days per week, and average reported use was 3.4 days per week. The authors also reported an 8.69% increase in pull requests per developer. These are findings from that study’s setting and methods, not an adoption target or a forecast for another engineering organization; GitHub authored the report.
METR’s 2025 randomized trial covered 16 experienced open-source developers and 246 tasks in their own mature projects. In that setting, allowing the tested early-2025 AI tools increased task completion time by 19%, even though participants estimated a time reduction. The result illustrates why self-estimates and observed outcomes can differ; it is specific to those developers, tasks, projects, and tools, and should not be generalized to every team.
DORA’s 2025 report presents modeled estimates for a 25% increase in AI adoption, not guaranteed effects observed in every organization. Its estimates include a 2.2% increase in productivity, a 2.1% increase in job satisfaction, a 0.4% increase in flow, a 2.6% decrease in time doing toilsome work, a 2.6% decrease in time doing valuable work, and a 0.6% decrease in software delivery performance. DORA presents 89% uncertainty intervals. Read these as uncertain modeled estimates from the report, not as a promise that increasing adoption will produce those changes in your teams.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Review adoption and outcomes together
Set a regular team review cadence and bring adoption signals, outcome measures, and developer feedback into the same conversation. DORA’s 2025 report describes metrics as tools that “drive conversations, support decisions, and help teams prioritize improvements.” Use the review to investigate a result, not to rank individuals by telemetry.
Recommended Free Tools
- Low adoption: Check access, onboarding, feature awareness, and workflow fit before drawing conclusions about value.
- High adoption without better outcomes: Examine task mix, quality and review burden, operational bottlenecks, and whether the tool fits the work being measured.
- Better throughput with worsening quality or stability: Treat the trade-off as part of the result, not as a secondary detail.
- Positive survey feedback without a system-level change: Keep the experience finding, but do not present it as measured productivity improvement.
After identifying a plausible friction point, adjust enablement or workflow and continue measuring with the same definitions. This turns the metrics into a feedback loop rather than a one-time verdict.
Quick Recap
A practical measurement plan
- State the decision and scope. Name the team or population, tools and features, workflow, and intended benefit.
- Define adoption and outcomes. Specify eligible users, activity windows, outcome definitions, quality and stability guardrails, and survey questions.
- Capture a baseline. Record the measures and context before rollout, using definitions that can be repeated later.
- Choose the comparison. Prefer random assignment or a staged rollout when practical; otherwise compare similar teams or tasks and document confounding changes.
- Join and validate data. Align user, team, and time-period records; document denominators and any gaps in feature coverage.
- Review evidence together. Examine telemetry, outcomes, and developer feedback, with sample size, period, exposure, and uncertainty visible.
- Make a bounded decision. Adapt access, training, or workflow based on what the evidence supports, then repeat the measurement.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
