A self-healing test can stay green while quietly checking a different element or behavior than its author intended. Luthfi Ferdian’s case for “throwaway automation” is not that maintained tests are obsolete: it is that some short-lived checks may be better kept as clear test cases and run by a browser agent, then promoted into reviewed, deterministic Playwright tests only when their value proves durable.
Why a green self-healed test may not be a valid test
When an application changes, a locator repair can help a test keep working. But finding a plausible replacement element is not the same as confirming the original scenario still works. If a repaired locator points somewhere else, the run may pass while no longer checking the intended behavior. That is the risk at the center of Ferdian’s argument, not evidence that every self-healing product behaves this way.
As an Amazon Associate I earn from qualifying purchases.
Treat an automatic repair as a proposed test change. Review what it now selects and whether the assertion still expresses the original requirement. A green result alone cannot establish that the test remains meaningful.
Ferdian’s point is not that scripts have no value. He writes, “You can’t break a script that doesn’t exist.” The practical question is whether a scenario is worth the cost of turning it into a maintained script now, or whether a well-specified, temporary check is enough.
#1 Best Overall
What throwaway automation means
In this approach, the test case is the durable asset: a readable description of setup, actions, expected outcomes, and evidence to capture. An agent uses a browser-control tool to carry out that case, and a human reviews both the report and what the agent actually observed.
This is a different trade-off from a conventional automated test. A script encodes repeatable actions and assertions; an agent-run case relies on instructions, available test data, and a reviewer’s judgment. The latter may be useful for a one-off investigation or release-specific check, but it does not automatically provide the repeatability or assurance of a maintained test.
How to write an agent-run test case
Make each expected result observable and unambiguous. For example, a cart check could specify:
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
- Preconditions: Use a logged-in standard user account and start with an empty cart.
- Actions: Add the promotion SKU, then open the cart.
- Expected results: A promotion banner appears, the discount text is visible, no error toast appears, and the mobile layout has no overlap.
- Evidence: Ask for step-by-step observations, a PASS or FAIL result for every expected outcome, and a screenshot.
This is an illustrative case, not a report of an executed test. Its usefulness depends on a seeded account, known test data, clear staging constraints, and instructions that let a reviewer tell success from a plausible-looking detour.
Playwright MCP provides a way for an agent to interact with a page using structured accessibility snapshots; its documentation also describes screenshot tools and headless operation. The snapshot is a structured representation for interaction, while a screenshot can help a person inspect visual layout. Neither tool capability proves that the agent interpreted the task correctly or that its run is deterministic. See the Playwright MCP repository and the official Playwright MCP introduction.
When an agent-run check may fit
Ferdian presents agent runs as candidates for checks whose useful life may be limited, rather than as a universal replacement for test suites. Examples include exploratory work, release-specific verification, a migration or refactor check, and reproducing a reported bug. These uses make sense when the check is worth doing but may not justify a permanent script.
Before choosing this route, weigh the scenario against the constraints that matter:
- Lifetime and recurrence: Is this for one release or investigation, or will it protect behavior repeatedly?
- Criticality: Is it exploratory, or does it cover a core user flow or important audit evidence?
- Repeatability: Is a human-reviewed observation sufficient, or must the result be stable enough to gate a merge or run across a large regression suite?
- Assertion clarity: Can the expected result be stated as a strict, observable condition?
- Evidence quality: Can a reviewer independently judge the steps and outputs?
- Environment control: Can the team provide seeded users, test data, and reliable staging conditions?
- Maintenance economics: Is the cost of building and maintaining a script justified compared with repeated agent runs and the uncertainty they bring?
This is a decision aid, not a validated scoring system. Ferdian supplies no measured break-even point for choosing an agent run over a script.
Where deterministic tests still belong
Ferdian does not position agent runs as merge gates or as a way to execute a thousand-test regression suite. He identifies non-determinism, slower execution than compiled scripts, and token costs as limits. Core flows, high-volume regression, and audit or compliance trails call for durable, repeatable automation when that evidence matters.
Apply the same caution to high-consequence behavior. For regulatory, financial, access-control, or core transaction scenarios, the risk of a false pass makes explicit assertions and durable controls especially important. That is a practical application of the distinction between exploratory checks and critical repeatable tests, not a measured outcome reported by Ferdian.
An agent can also work around a broken flow and claim success. Specify what must happen—not merely a goal such as “complete checkout”—and reject a pass if the expected result is missing. Inspect the path taken and the captured evidence. Do not keep rerunning a failing check until it happens to turn green; a real failure should remain visible for investigation.
Free tools Windows power users keep installed
One-click scans. No signup required.
When to promote a temporary check into Playwright
Promote a scenario when it recurs, catches meaningful defects, or protects critical behavior. Promotion means writing and reviewing a script with explicit assertions, rather than preserving an agent transcript and treating it as durable coverage.
Best Value
For the illustrative cart case, a maintained Playwright test could use explicit API setup and assert the expected promotion state through a test identifier. The point of this lifecycle is to let temporary execution help reveal which checks deserve ongoing investment; stable, important behavior earns reviewed automation.
Ferdian’s rule of thumb is, “if you wouldn’t urgently fix a script when it breaks, don’t promote it.” He also cautions that “Your value as an SDET isn’t how many scripts you maintain.” Both are opinions about prioritization, not claims that scripts or maintenance are unimportant.
What the evidence does—and does not—show
Ferdian’s article and the Playwright documentation describe a workflow and tool capabilities; they do not provide a measured comparison of false-positive rates, maintenance hours, execution speed, token costs, or defect detection between agent-run checks and maintained suites. The case for throwaway automation is therefore a judgment about fit and maintenance trade-offs, not a demonstrated performance advantage.
Which part of your current suite could be replaced by a well-written test case and an agent, and what would you need to see before you trusted it? Read Luthfi Ferdian’s article.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
