There is no universal number of weeks for an A/B test. Work out how many eligible users you need to detect the smallest effect that would change your decision, estimate how quickly those users will enter the test, and choose a stopping method before launch. Then account for calendar patterns and delayed outcomes that could make a shorter run misleading.
What determines how long your A/B test needs to run?
Duration is the sample required for a decision, divided by the rate at which eligible users enter the experiment. The required sample depends on the primary metric, its baseline rate or variability, the effect you want to detect, and your statistical decision criteria. At the same traffic rate, detecting a smaller effect generally requires more observations and more time.
As an Amazon Associate I earn from qualifying purchases.
Amplitude’s key terms describe the relationship between sample size and traffic; Google Search Central likewise notes that duration varies with traffic and conversion rate. A calculator’s output is an estimate, not a guaranteed end date.
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Clear out junk files and repair common Windows errors3Scan for outdated or missing drivers - takes under a minutePlan the duration before launching
- Choose one primary metric. State what change you are testing and which outcome will decide whether to ship it. Name guardrail metrics separately so that a gain in the primary metric cannot hide a meaningful downside elsewhere.
- Set a decision-relevant minimum detectable effect. The MDE is the smallest effect worth detecting—not simply the most optimistic improvement you hope to see. If a smaller change would not alter your decision, planning for it can needlessly increase the sample and run time. Amplitude explains how to set an experiment’s MDE.
- Estimate the sample using the eligible population. Use a baseline rate or metric mean and variance that reflect the users who can actually enter the test, along with the intended allocation between variants. If only a subset of site visitors is eligible, total site traffic will overstate the exposure rate. Statsig’s power-analysis documentation describes the inputs used to estimate sample needs.
- Translate the sample into elapsed time. Divide the required exposures by expected eligible exposures per day, accounting for the allocation to each variant. Amplitude’s duration-estimation guidance describes estimates based on means, variances, and exposure rates; its estimate can be less reliable when those inputs change, and one workflow assumes constant daily exposure.
- Check whether the calendar affects the result. Consider weekday differences, seasonal changes, delayed conversions, and any learning period in the system being tested. Reaching the planned sample quickly does not make results representative if important behavior has not had time to appear.
- Choose the stopping rule before looking at results. Decide whether the test uses fixed-horizon analysis or a sequential method, and follow that method when reviewing interim data.
Fixed-horizon or sequential: which stopping method fits?
| Method | How interim reviews work | Planning commitment | Main caution |
|---|---|---|---|
| Fixed-horizon | Plan a sample target and evaluate results at the planned endpoint; ordinary repeated checks do not justify stopping early when a favorable result appears. | Prespecify the sample and stopping point. | Repeatedly checking and stopping on ordinary significance can inflate false-positive risk. |
| Sequential | Interim decisions can be valid when the sequential adjustment is active and the chosen decision criteria are followed. | Plan for interim review under the method’s decision rules. | Sequential analysis does not remove all risk or ensure that every guardrail has enough data to detect harm. |
Statsig’s sequential-testing documentation explains adjusted interim inference. Whichever method you choose, do not treat a favorable snapshot on a selected metric as proof that the experiment is adequately powered for other outcomes.
#1 Best Overall
When should calendar coverage change the estimate?
A sample-size target and an elapsed-time target answer different questions. Sample size addresses how much data the analysis needs; elapsed time helps ensure that data covers relevant patterns and that outcomes have time to mature. For example, a conversion metric with a delay needs time for users who entered near the end of the test to convert before the result is assessed.
Google Ads API documentation recommends running its campaign experiments for at least four weeks to cover weekly cycles, conversion delays, and learning periods. That guidance is specific to Google Ads campaign experiments, not a general duration rule for website or product A/B tests. See the Google Ads API experiment-reporting guidance.
Why can a duration estimate change?
An estimate depends on inputs such as eligible exposure rate, baseline metric behavior, and variance. If targeting changes, traffic shifts, or the metric behaves differently over time, the estimate may no longer describe the experiment you are running. Seasonality can also reduce the accuracy of forecasts based on past averages. Revisit the forecast when a material input changes, but do not use an updated estimate as a reason to stop a fixed-horizon test on an unplanned favorable result.
When is the test finished?
Finish when the evidence meets the stopping rule you selected and the result supports a practical decision—not merely because a dashboard briefly shows significance. Interpret the primary metric alongside the guardrails and allow for relevant outcome delays. For website tests, Google Search Central advises ending the test once enough data has been collected and removing test elements after the conclusion. Read Google Search Central’s website-testing guidance.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
