Use a small, visible retry allowance in CI only to absorb plausibly intermittent failures—not to make an unstable test suite look healthy. A test that fails once and passes on a retry is still flaky; preserve that result in reports and fix recurring causes. In local development, failing on the first attempt often gives the clearest signal.
What a retry does—and what it does not do
“Retry” can mean two different things. A framework may repeat a query or assertion while waiting for asynchronous UI state, or it may rerun an entire failed test. These solve different problems.
As an Amazon Associate I earn from qualifying purchases.
- Query or assertion retry: checks repeatedly for a condition, stopping when it becomes true or times out. This is usually the right response when a page or component needs time to render.
- Whole-test retry: restarts the test after failure, including its setup and other test work. It can provide another attempt after an intermittent failure, but may repeat side effects and does not fix a bad wait or assertion.
Cypress documents that its queries and assertions can retry while an action such as .click() runs once. For example, waiting for a button to become actionable is different from repeating a test that clicked it and then failed later. See Cypress retry-ability documentation.
Playwright distinguishes a test that passes on its first run from one whose first run fails but a retry passes: the latter is reported as flaky, not as an ordinary first-attempt pass. A test that remains unsuccessful through its retries is failed. Keep that distinction visible in CI. See Playwright test retries.
#1 Best Overall
When should you retry an end-to-end test?
Use a whole-test retry for plausible transient failures
A limited retry can be reasonable in CI when there is evidence the failure may be intermittent: a temporary service or network problem, a race around an animation or asynchronous response, or a constrained runner environment. The retry gives the run another chance without pretending the initial failure did not happen.
Cypress identifies animations, API calls, server or database availability, dependency availability, and network issues among possible sources of unreliable tests. Those are reasons to investigate a failure, not proof that any particular failure is harmless. See Cypress test performance guidance.
Do not use retries to mask deterministic failures
A reproducible product defect, stale selector, invalid test data, shared state, or incorrect assertion is unlikely to be repaired by rerunning the same scenario. Diagnose and correct the test or product behavior instead. This is a practical application of framework guidance on isolation and meaningful assertions; the official documentation does not provide a single exhaustive decision tree for every failure.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Clear out junk files and repair common Windows errors3Fix the driver behind crashes, sound loss and screen glitchesPrefer condition-based waits for asynchronous UI
If the test needs a page element, response, or state transition, wait for that condition with the framework’s retryable query or assertion rather than sleeping for an arbitrary duration. Cypress recommends retryable assertions instead of numeric waits in the situations its guidance covers; Playwright recommends auto-retrying assertions for asynchronous pages. See Playwright assertions.
How many retries should you allow?
There is no universally correct retry count. Start with the smallest allowance that addresses a real CI workflow problem, and evaluate it against your suite’s failure patterns and execution cost. Broadly raising the count can multiply test work while making recurring flakiness less visible.
| Policy example | What the official guidance establishes | Useful interpretation |
|---|---|---|
| Playwright default | Retries are disabled by default. Playwright documentation | Enable retries deliberately rather than assuming the runner already retries. |
| Cypress example | Cypress performance guidance illustrates one retry in run mode and none in open mode. Cypress documentation | This is a framework-specific example, not a standard for every suite. |
These documented examples are not interchangeable prescriptions. Defaults and configuration syntax may change, so check the current documentation for the runner version in your project before setting exact values.
Choose a policy for local runs and CI
- Local development: consider zero whole-test retries so a failure is immediate and easy to reproduce while editing.
- CI: consider a modest retry allowance when transient failures otherwise trigger unnecessary manual reruns. Preserve the first failure and report the recovered test as flaky.
- Release qualification or suite-health measurement: consider separately gating tests with any failed attempt, even if a retry passes. Cypress documents an experimental strategy for this behavior; verify its current status and configuration before relying on it. See Cypress test retries.
Choose based on the purpose of the run: continuity, failure signal, and the time spent executing and diagnosing retries are competing concerns.
Make whole-test retries safe and informative
- Isolate test data and state. Make each test independent and give it the state it needs so another test or a retry cannot contaminate the result. Playwright recommends isolation so tests can be run and retried independently. See Playwright best practices.
- Make setup and cleanup repeatable. Before enabling whole-test retries, confirm that hooks and test actions can safely run again. Consider whether an attempt may have created records, sent messages, or changed external state before failing.
- Assert on user-visible behavior. Check that the outcome a user depends on has occurred, rather than relying on timing assumptions or an implementation detail that can change.
- Keep attempt-level evidence. Retain the original assertion output and retry count. Capture screenshots or video where useful, and traces for failures. Playwright recommends its Trace Viewer for CI failures and documents configuring traces on the first retry; Cypress documents retry-specific screenshot and video handling. See Playwright Trace Viewer and Cypress test retries.
- Review trends and repair repeat offenders. Track which tests retry and how often. Cypress Cloud documents flaky-test management for inspecting tests with high flake rates; that hosted capability is product-specific. See Cypress Cloud flaky test management.
Cypress documentation characterizes frequently retrying tests as technical debt to fix, rather than a permanently acceptable state. A green final status should not erase the evidence that a test needed another attempt.
How to decide what to do after a retry
| Observed result | Recommended response |
|---|---|
| First attempt fails; retry passes | Keep the test visible as flaky. Review the failure evidence and track whether it recurs. |
| Attempts fail in the same reproducible way | Investigate the assertion, selector, test data, or product behavior; do not increase retries as the first fix. |
| Failures vary with network, dependency, or runner conditions | Check service and infrastructure health, then decide whether a limited CI retry is justified while the underlying instability is addressed. |
| Test passes only after a long arbitrary wait | Replace the timing guess with a retryable assertion for the specific condition the test needs. |
| A retry repeats or duplicates a side effect | Make the test data and setup safe to repeat, or avoid whole-test retries for that scenario. |
Or skip the browser setup
If you need a screenshot to inspect a failed page or attach visual evidence to a debugging workflow, ScreenshotNeo can return an image with one request. Its capture can accept a cookie or consent banner like a visitor and remove more than 60 known consent platforms, newsletter popups, and chat widgets; each cleanup step can be turned off. Bot checks or CAPTCHAs, blank pages, timeouts, failed loads, and cache hits cost nothing, and responses indicate the page verdict and billing status in headers. Its MCP server provides take_screenshot, get_page_info, and capture_pdf tools for AI agents and other MCP clients.
Example cURL request (replace the target URL and API key):
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webpSee the ScreenshotNeo API documentation for parameters and response details. One thousand screenshots per month are free with no card; paid plans start at $5 for 3,000. Sign up for ScreenshotNeo’s free plan.
Recommended Free Tools
Performance and reliability trade-offs
Every whole-test retry consumes execution time, and a high retry count applied broadly compounds that overhead. It can also make the final CI status less informative if the first failure is hidden. A narrow assertion retry is often a better fit for delayed UI state; a whole-test retry is broader because it repeats the scenario and its hooks. Track retry frequency alongside overall pass/fail status so the team sees both workflow continuity and test health.
Frequently Asked Questions
Should a test that passes on retry count as passing?
It may leave the run green under your CI policy, but it should remain identifiable as flaky because its first attempt failed.
Should I retry tests in every local run?
Not necessarily. Zero whole-test retries locally can make failures easier to notice and reproduce; choose local and CI behavior separately.
Can retries fix flaky tests?
No. They can provide another attempt, but repeated recovery is evidence to investigate synchronization, state isolation, application behavior, or infrastructure.
Free tools Windows power users keep installed
One-click scans. No signup required.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
