A Playwright test that fails and then passes on retry is classified as flaky—but that label reports inconsistent results, not a repaired test. Retries can help reveal the pattern and capture evidence; the durable fix is to find whether the problem is a timing assumption, a weak locator, shared state, or an application behavior that changes between runs.
What a passing retry tells you—and what it does not
Playwright’s test runner marks a test flaky when it fails on an attempt and passes on a retry. That is useful information: the result depends on something that varies between attempts. It does not identify the cause or prove the underlying behavior is reliable. Treat the first failure as the evidence to investigate, rather than letting the later green result end the investigation. Playwright’s retry documentation explains the classification and retry behavior.
As an Amazon Associate I earn from qualifying purchases.
Retries are worth configuring deliberately, especially when they help expose intermittent failures or trigger trace capture in CI. They should not be the only remedy. A test that passes only after another attempt remains harder to trust and diagnose than an independent test that succeeds consistently.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Diagnose the failure in a useful order
- Separate first-attempt results from retry results. Use the test report to determine whether the failure is consistent or intermittent. A retry pass identifies flakiness, not its root cause. Record which attempt failed and inspect that failure before changing retry counts.
- Match the assertion to asynchronous UI behavior. Read the failed expectation and check whether it samples a value only once. Playwright’s web-first assertions, such as
await expect(locator).toBeVisible(), re-check until the condition succeeds or times out. A manual immediate boolean assertion can run before an asynchronous UI update. The documented default expect timeout is five seconds; it can be configured in the test configuration or for an individual assertion. See Playwright’s assertion guidance. - Inspect the locator and the action that failed. Prefer locators that express the user-facing contract—such as role and accessible name, label, or meaningful text—when those describe the target. A test ID can be appropriate when you intentionally define a test-facing contract. Avoid selectors coupled to incidental implementation details when a more meaningful locator is available. Playwright documents these locator choices at Locators and Best practices.
- Check what Playwright was waiting for before interacting. For a click, Playwright checks that the locator resolves to one element and that it is visible, stable, enabled, and able to receive events. If an action times out, one of those conditions may never have become true; inspect the timeout rather than assuming the fix is simply to wait longer. These checks and their meanings are documented in Actionability.
- Test independence and hidden shared state. Run the failing test by itself, then consider whether it relies on another test’s data, order, cookies, or local storage. Playwright recommends independent tests, and each test gets an isolated browser context with its own browser state. Make setup self-contained so a test can run alone and in a different order. See Best practices and Browser contexts.
- Use a CI trace to inspect the failing attempt. Once the likely class of problem is clearer, capture a trace on retry and review its action timeline and DOM snapshots. They can show whether the UI was still changing, a click was intercepted, the expected element was absent, or the page reached an unexpected state. Then change the cause suggested by that evidence and verify the original failure condition again.
Why action waits and assertion waits are different
Actionability checks happen before an interaction: they help ensure a click targets an element that is ready to receive it. A web-first assertion waits for the resulting state you expect, such as a confirmation becoming visible after submission. Passing the actionability checks does not establish that the application completed the expected change; likewise, waiting for the expected state does not make a poorly targeted click correct.
#1 Best Overall
Do not use force: true as a routine way to make a click pass. It disables non-essential actionability checks, including whether the target receives events, and can hide an overlay or interaction problem instead of fixing it. For complex conditions that cannot be expressed as a web-first locator assertion, Playwright documents expect.poll and expect.toPass. Configure toPass deliberately: its default timeout is zero and it does not use the custom expect timeout. The relevant behavior is described in the assertions documentation.
Why a test can pass locally but fail in CI
A local pass establishes only that the test passed in that run’s conditions. It does not rule out a race, order dependency, or state assumption that becomes visible under another run. Start with the evidence: the failed action, the assertion’s timing, whether the test passes alone, and what the trace shows. These checks can distinguish a delayed UI update from a locator that resolves differently or a test that expects state created elsewhere.
Do not assume a particular worker count, timeout, or retry setting is a universal CI fix. Playwright’s documentation describes the mechanisms and configuration options, not the cause in a specific application. Adjust a timeout only when evidence indicates the condition is legitimate but needs more time; otherwise a longer timeout can postpone the failure without removing the race.
Capture traces where they are useful
Playwright recommends recording traces on the first retry in CI, so the first failure can be inspected when the runner retries the test. The Trace Viewer lets you review action timing and DOM snapshots. Recording every test’s trace can be performance-heavy, so choose a capture policy that preserves useful failure evidence without paying that cost indiscriminately.
Trace configuration is version-sensitive and supports options including on-first-retry, on-all-retries, and retain-on-failure. Check the documentation for the version you run before changing the setting. See Trace Viewer and Test configuration.
Choose a fix that addresses the cause
| Candidate change | What it improves | What to verify |
|---|---|---|
| Replace a one-time check with a web-first assertion | Waits for an expected UI state that updates asynchronously. | That the assertion describes the user-visible result, not merely a delay. |
| Use a role, label, accessible name, or meaningful text locator | Aligns the test target with the user-facing interface. | That the locator identifies the intended element uniquely and reliably. |
| Make setup self-contained | Reduces dependence on another test’s state or execution order. | That the test passes alone and remains independent of run order. |
| Increase a timeout | Allows a legitimate condition more time to occur. | That the trace supports a slow-but-correct transition; a longer wait alone does not establish a fix. |
| Use a forced action | Bypasses some actionability checks. | Whether it conceals an intercepted event or other interaction defect; it is not a routine reliability fix. |
| Keep retries and capture a trace on retry | Surfaces intermittent status and preserves diagnostic evidence. | That the first failed attempt is investigated rather than treated as repaired by the pass. |
A strong fix makes the test’s assumptions explicit, preserves useful actionability checks, and leaves enough evidence to understand a future failure. Confirm it against the same condition that originally failed, then run the test independently and as part of the suite.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
