Yes. Automated UI tests can be flaky: the same test may pass once and fail the next time even when the relevant application code has not changed. A single result is therefore not a reliable signal of a regression until you understand what differed between attempts. The practical fix is to identify the changing condition—such as UI state, request timing, test data, execution order, a dependency, or CI load—and address it directly.
What “unstable” means in UI testing
A flaky test produces inconsistent results under conditions that should be equivalent. It might fail on the first run and pass on retry, pass by itself but fail in the suite, or behave differently in CI than on a developer’s machine. Playwright labels a test that fails and then passes on retry as “flaky”; retries are off by default in Playwright. Playwright’s retry documentation explains the distinction between first-attempt passes, flaky tests, and failures.
As an Amazon Associate I earn from qualifying purchases.
A retry pass does not prove the application is healthy, nor does the initial failure prove there is a product regression. Treat both attempts as evidence to investigate. A useful triage question is: “Are these failures real regressions, or known flakiness?”
Free tools Windows power users keep installed
One-click scans. No signup required.
Common causes and the fixes that match them
Timing and asynchronous UI updates
Browser actions and application updates do not happen in a single synchronous step. An animation, API response, database operation, server startup, network delay, or dependent resource can shift when an element appears or when the page reaches its final state. A test may click too early, assert before a response updates the page, or observe a temporary DOM state.
Wait for a condition tied to the behavior being tested, then assert the expected result. Playwright checks actionability before supported interactions: for example, a click target must resolve to one element and be visible, stable, unobscured, and enabled. Its assertions can retry until the expected condition is met. See Playwright auto-waiting and Playwright assertions.
With Selenium, choose an appropriate condition-based wait. Its documentation warns that mixing implicit and explicit waits can lead to unpredictable timeout durations. Fixed sleeps are a poor default: a short sleep may still be too short, while a long one wastes time on every run. See Selenium waiting strategies.
Shared state, test data, and execution order
A test can pass alone and fail in a suite because another test changed a shared record, left data behind, reused an output file, or ran in an order the first test implicitly expects. Parallelism can expose these dependencies. Playwright browser contexts isolate browser-side state for workers, but that does not automatically isolate backend records, files, or other shared resources. Its guidance recommends independent tests and distinct backend data. See Playwright parallelism and Playwright best practices.
Windows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallCrashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minute- Create the data each test needs and clean it up deliberately.
- Use unique identifiers for records and generated files.
- Do not make one test’s success or cleanup a prerequisite for another.
- If a resource cannot be isolated, control access to it explicitly and document the concurrency limit.
Brittle checks and implementation details
A test tightly coupled to incidental markup or internal implementation details may fail after a harmless refactor even though the user-facing behavior is unchanged. Assert the visible outcome that matters to the scenario, rather than a detail that users do not depend on. For asynchronous changes, use an assertion that waits for the expected state instead of checking once at an arbitrary instant. Playwright’s best-practice and assertion guides describe these approaches: best practices and retrying assertions.
External services, networks, and CI variation
A third-party page or API, an unstable network, an unavailable test service, or a resource-constrained CI runner can cause intermittent failures. When the scenario is about your application, control third-party responses where practical instead of making the test depend on a service your team does not control. Use a stable staging environment and consistent test data. Cypress identifies networks, servers, databases, and resource dependencies among potential race sources in its test-retry guidance.
If the test is reliable locally but intermittent in CI, compare the failing and passing attempts before increasing timeouts globally. Check service availability, runner resource pressure, browser and operating-system differences, and collisions in test data. Cypress Cloud’s replay and flaky-test documentation describes using DOM state, network requests, console logs, and element state around a failure to investigate timing, race, and environment problems: Cypress Cloud flaky-test management.
Rank #4
A practical diagnosis sequence
- Preserve the original failure. Reproduce it without changing the test, application code, or environment. Record the exact failing step and whether a retry passes; do not let a green retry erase the first failure from reports.
- Compare one passing attempt with one failing attempt. Look for a late or different request, an element that was moving or covered, an unexpected DOM state, overlapping data, or a CI-only service or resource condition.
- Change the execution context deliberately. Run the test alone, then with its surrounding suite or parallel workers. If only the suite run fails, investigate ordering, cleanup, and shared backend state.
- Fix the cause, then verify under the conditions that exposed it. Rerun enough times in the relevant suite or CI context to check that the symptom is gone. Keep any small retry allowance as a diagnostic safety net, not as proof of stability.
Retries: useful evidence, not a permanent repair
Retries can help expose an intermittent failure and can sometimes recover from a transient CI interruption. They also add execution time, and repeated retry success can hide an unreliable test. Playwright distinguishes retry-pass tests as flaky and has retries off by default; Cypress likewise documents retries as a way to detect flaky behavior. Cypress recommends keeping retry counts low and using flake data to locate root causes; each retry reruns the test and hooks. See Playwright retries, Cypress test retries, and Cypress test-performance guidance.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →Track recurring retry passes separately from first-attempt passes. Preserve the first failure’s logs, screenshots, trace, or replay material where available, and prioritize repeated failures rather than quietly accepting them as normal.
Best Value
What published evidence can—and cannot—tell you
A 2025 IEEE ICST empirical study examined 49 web projects and 123 DOM-event-related test cases. Within that dataset and scope, it reported these observed repair-strategy shares:
| Observed repair strategy | Share reported by the study |
|---|---|
| DOM interaction synchronization | 50.4% |
| Conditional waits for event completion | 38.2% |
| Consistent DOM state transitions | 11.4% |
These figures describe repair strategies observed in that study, not the prevalence of all UI-test flakiness or a guarantee that synchronization is the cause of any particular failure. They do support checking interaction timing and state transitions early. The study is “An Empirical Study of Web Flaky Tests: Understanding and Unveiling DOM Event Interaction Challenges,” IEEE ICST 2025.
ScreenshotNeo for repeatable visual evidence
When visual inspection of a page is part of debugging, a screenshot can help compare what the browser showed on passing and failing attempts. ScreenshotNeo is a website screenshot API and MCP server from Yorker Media. It can capture PNG, JPEG, WebP, or PDF; this is an optional way to collect page evidence, not a substitute for fixing test synchronization, data isolation, or CI conditions. Visit ScreenshotNeo for the service overview.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Or skip the browser setup
A GET request can capture a URL directly. Replace the example URL and provide your ScreenshotNeo API key:
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webpSee the ScreenshotNeo API documentation for request options. Cookie banners, popups, and chat widgets are removed before the shot; those cleanup steps can be turned off. Bot checks, blank pages, timeouts, failed loads, and cache hits are not billed, and each response reports its page verdict and billing status in headers. An MCP server provides take_screenshot, get_page_info, and capture_pdf tools for AI agents and MCP clients. The Free plan includes 1,000 screenshots a month without a card; paid plans start at $5 for 3,000 screenshots.
Sign up for 1,000 free screenshots a month—no card required.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
