The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Use Bonferroni when your priority is controlling the chance of even one false positive across a defined set of tests. Use Benjamini–Hochberg (BH) when screening many hypotheses and you can accept control of the expected proportion of false findings among the results you report. The choice is about which error rate fits the decision—not simply how strict you want to be.
What each method controls
The family-wise error rate (FWER) is the probability of making at least one false rejection in a family of hypotheses. The false discovery rate (FDR) is the expected proportion of false rejections among all the hypotheses rejected. If every null hypothesis is true, FDR equals FWER; when some nulls are false, FDR can be smaller.
As an Amazon Associate I earn from qualifying purchases.
Consequently, an FDR target such as q = 0.05 does not mean that each reported finding has a 5% chance of being false. It controls an expected proportion across the rejected findings, not the probability that any particular finding is false.
Free tools Windows power users keep installed
One-click scans. No signup required.
Choose based on the cost of a false positive
| Analysis situation | Method to consider | Why |
|---|---|---|
| A small, planned family of primary or confirmatory comparisons where one false claim would be costly | Bonferroni, or Holm if FWER control is required | Targets the probability of one or more false rejections across the family. Holm also controls FWER and can be less conservative than plain Bonferroni. |
| A large discovery screen where results will be followed up and some false leads are tolerable | Benjamini–Hochberg | Targets the expected proportion of false discoveries among rejected hypotheses and often retains more power when effects exist. |
| Tests may be dependent and the dependence structure is unclear | Do not assume ordinary BH is valid automatically | The original BH guarantee assumes independent test statistics; documented extensions cover certain positive-dependence structures, not arbitrary dependence. |
| A regulatory, clinical, or other high-consequence decision | Follow the prespecified analysis plan and applicable field guidance | The study design and governing standards may define the relevant multiplicity family and error target. |
Consider four things before choosing: the error rate you need to control, the relative costs of false positives and missed effects, how the hypothesis family is defined, and the dependence assumptions the method requires. More power is not automatically preferable if it delivers the wrong guarantee for the decision.
#1 Best Overall
How Bonferroni works
For a family of m tests and a chosen family-wise level α, compare each raw p-value with α/m. Equivalently, calculate an adjusted value for each test as pi,adj = min(1, m × pi) and compare it with α. For example, with α = 0.05 and 10 tests, the per-test cutoff is 0.005. This is an arithmetic example, not a study result.
Bonferroni controls FWER without requiring independent tests, provided the individual p-values are valid and the family is defined appropriately. NIST describes the method for finite sets of selected contrasts and simultaneous confidence limits with coverage of at least 1 − α (NIST Engineering Statistics Handbook: Bonferroni method; NIST Engineering Statistics Handbook: multiple comparisons).
How Benjamini–Hochberg works
BH is a step-up procedure. Sort the m p-values from smallest to largest, then compare the value at rank i, written p(i), with (i/m)q, where q is the chosen FDR target. Find the largest rank k for which p(k) ≤ (k/m)q. Reject the hypotheses corresponding to ranks 1 through k. If no ranked p-value meets its threshold, reject none.
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchBecause it is step-up, a qualifying p-value at a higher rank can lead to rejecting all smaller-ranked p-values too. The original Benjamini–Hochberg paper introduced FDR control as an alternative to controlling FWER and gives the procedure under independence (Benjamini and Hochberg, 1995).
Rank #3
Dependence can change whether ordinary BH is appropriate
Tests are often related, but correlation alone does not establish that ordinary BH has its intended guarantee. The original result assumes independent test statistics. The What Works Clearinghouse Procedures Handbook, Version 4.0, discusses applicability under certain positive-dependence conditions; that is not a general guarantee under every dependence structure (WWC Procedures Handbook, Version 4.0).
If the dependence structure is uncertain or does not meet the conditions for ordinary BH, investigate a method justified for that setting, such as Benjamini–Yekutieli, and consult guidance for your field. R documentation describes BH and BY as FDR methods and notes Holm’s FWER control and dominance over unmodified Bonferroni (R documentation for p.adjust).
Rank #4
Define the hypothesis family before interpreting results
The correction depends on what counts as one family of tests. Relevant tests may include multiple endpoints, contrasts, outcomes, subgroups, or analyses that could have supported the claim being made. Explain why the family was selected; a correction cannot repair a family defined after looking for favorable results.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
These methods also assume the underlying tests produce valid p-values under their null hypotheses. Multiplicity adjustment does not fix a misspecified model, biased sampling, p-hacking, or invalid p-values. NIST’s examples concern a finite set of contrasts selected in advance (NIST Engineering Statistics Handbook: Bonferroni method).
Report enough detail to make the choice interpretable
- Define the family and state how many hypotheses it contains.
- Name the procedure and the target α or q.
- Give adjusted p-values or adjusted confidence intervals, as appropriate.
- Explain in words whether the analysis controls FWER or FDR.
- Say whether the analysis and family were prespecified or exploratory.
- For ordinary BH, state the dependence condition relied on.
For confirmatory or regulated work, use the error target and multiplicity family specified by the protocol or applicable standards rather than choosing a method solely for its apparent power.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
