AI can make some test-case work much faster and cheaper, but the strongest direct comparison available is a proof of concept for generating synthetic survey data—not an industry-wide measure of test design. In that study, manual creation was estimated at 8 hours and $381 per case; two AI workflows took 16.5 minutes or 3.75 minutes and were each estimated at $0.10 per case. Those AI figures excluded the cost of building and deploying the system and training a person to create the manual cases.
What the direct cost comparison measured
A 2025 National Cancer Institute (NCI) study tested ways to generate synthetic answers for surveys in the CHARMS Rasopathy workflow. The aim was to let the team run existing automated tests without using identified patient-level production data. For the manual process, testers traversed a survey, copied its questions into an input file, and created answers. The automated workflow extracted questions from survey JSON, used a persona and question dependencies to generate synthetic answers, then packaged the results for testing. The NCI study reported these per-case estimates:
As an Amazon Associate I earn from qualifying purchases.
| Approach in the NCI study | Estimated time per case | Estimated cost per case | What the comparison represents |
|---|---|---|---|
| Manual case generation | 8 hours | $381 | Authors’ estimate based on an average automation tester salary of $99,000 |
| Azure OpenAI GPT-3.5 | 16.5 minutes | $0.10 | One of the study’s synthetic survey-answer generation workflows |
| Self-hosted AWS Flan T5-XL | 3.75 minutes | $0.10 | One of the study’s synthetic survey-answer generation workflows |
The authors attributed GPT-3.5’s longer elapsed time partly to waiting between API calls; the self-hosted endpoint did not require that wait. The $381 manual estimate depends on the study’s salary assumption, while the AI model names and $0.10 estimates describe that study’s configuration—not current software or cloud price quotes.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Fix the driver behind crashes, sound loss and screen glitches3Repair Windows errors before they cause bigger problemsWhy the headline savings are not a universal benchmark
The NCI authors wrote: “Synthetic data generation is greater than 3,000X cheaper and greater than 120X faster than the manual test case generation process”. That comparison applies to their particular synthetic survey-data workflow and per-case calculation. It excludes the cost of building and deploying the LLM workflow and the time needed to train a person to answer the survey manually. It therefore does not establish a total cost of ownership or a savings rate for every testing team.
The study generated 50 cases with each of two AI approaches. Its survey had conditional paths, and the authors said exhaustive coverage of every possible path was impractical. They assessed efficiency, effectiveness, and realism using measures such as questions answered, response complexity, demographic coverage, and clinical expert review. The generated data represented some categories better than the manually created data, but also had demographic omissions. Lower generation effort alone does not prove complete coverage or realistic test data.
What other studies say—and what they do not
Other empirical work examines related but distinct tasks. Its results are useful for understanding lifecycle effort, but should not be merged with the NCI cost figures into a single AI-savings estimate.
Writing and evolving executable web tests
Leotta, Ricca, Marchetto, and Olianas compared NLP-based web testing with Selenium WebDriver and Selenium IDE across nine suites on different web applications. Three junior testers and developers, each with roughly two to three years of end-to-end web-testing experience, took part. The study considered initial script development, reuse after an application version change, suite-evolution time, and cumulative effort. The authors concluded that NLP-based testing appeared competitive for the small-to-medium-sized suites they examined and could reduce combined development and evolution effort. This is a lifecycle-cost perspective on test automation, not a general LLM cost percentage. Read the study.
Windows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallCrashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteGenerating scripts from written test cases
A 2024 preliminary study evaluated ChatGPT and GitHub Copilot for developing web end-to-end test scripts from natural-language descriptions. It reported lower development time when the cases were clearly defined in Gherkin, while noting that testers need enough scripting skill to modify AI-produced code. The public repository record does not provide a numeric breakdown, so it cannot support a specific percentage saving. View the repository record.
Designing system tests from user stories
A 2025 public-sector study describes a GPT-4 tool connected to Redmine and Squash TM. Analysts said the tool reduced effort, and the study reports that generated and manually designed tests had the same functional coverage. The accessible page gives no quantified time or money comparison. Read the study.
Executing manual tests with visual assistance
A study by Bauer, Frattini, and Alégroth involved 13 professionals from six companies using Augmented Testing, a visual support layer for manual GUI regression work. Mean execution time across all tests fell from 1,222 seconds to 779 seconds, a 36% reduction. Six of eight cases were faster with assistance; the two shortest slightly favored the baseline. This measured the execution of existing manual tests, not the cost of creating test cases. Read the study.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.How to calculate your team’s real cost
For a useful local comparison, count what it takes to produce an accepted, maintainable case—not just the model’s generation time or usage charge. Apply the same scope and quality checks to manual and AI-assisted work.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
- Setup and integration: Include prompt or framework development, data preparation, connections to test systems, deployment, and staff onboarding. The NCI per-case calculation excluded framework build and deployment.
- Human review and correction: Record time to check accuracy, repair errors, verify expected results, and ensure the case is usable. AI-generated scripts may require a tester with enough coding skill to modify them.
- Coverage and realism: Compare functional and branch coverage, boundary cases, realistic data, and defect detection—not merely how many cases were generated. The NCI study found demographic omissions; the public-sector study reported matching functional coverage but no quantified time comparison.
- Maintenance: Measure how many cases survive application changes and the time needed to update the rest. The Leotta study explicitly considered reuse and suite evolution.
- Labor and tool costs: Use your organization’s loaded labor rate and current software or cloud charges. The NCI figures reflect its stated salary assumption and study configuration, not your team’s rates.
- Scale and release cadence: Track the number of cases and releases over which setup, review, and maintenance costs are spread.
A practical scenario calculation is: total effort and cost = initial setup + per-case generation + human review and correction + maintenance + tool or cloud charges, measured across the expected number of cases and releases. Compare that with the equivalent manual workflow. AI assistance becomes economically attractive when the recurring effort it saves outweighs setup, review, and maintenance at your team’s scale.
When AI assistance may pay off
The strongest case is a structured, repeatable task where a working generation workflow can be reused and people can efficiently verify its output. The NCI result shows how low marginal effort can be in one such task; it does not show that every test-design activity can be reduced to a few minutes or cents. For changing applications, complex expected results, or strict coverage requirements, review and maintenance may dominate the calculation. Measure time per accepted case and the quality of the resulting suite before deciding whether the savings hold for your process.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
