Some links on this page are affiliate links: if you buy through them we may earn a commission, at no extra cost to you.
A statistic can be calculated correctly and still support a misleading conclusion. The number may lack context, hide uncertainty, describe an unrepresentative sample, or be presented in a way that encourages readers to infer more than the evidence shows.
The “seven deadly sins” are a useful teaching framework for spotting common interpretation errors—not a formal statistical standard or a complete list of every way data can mislead. The framework appeared in a 2017 article by Winnifred Louis and Cassandra Chapman for The Conversation. Here is a fuller guide to the seven errors, the questions that help expose them, and several related pitfalls to watch for.
The seven errors at a glance
| Error | What goes wrong | Ask this |
|---|---|---|
| 1. Treating small differences as meaningful | Ordinary uncertainty is mistaken for a real change. | How large is the difference, and how uncertain is the estimate? |
| 2. Confusing statistical and practical significance | A detectable effect is treated as important—or an uncertain result as proof of no effect. | What is the effect size, and would it matter in context? |
| 3. Ignoring extremes | An average is assumed to describe everyone. | How are the distribution, tails, and relevant subgroups affected? |
| 4. Trusting coincidence | A pattern discovered after looking through data is treated as a reliable finding. | Was it predicted, tested fairly, and replicated? |
| 5. Getting causation backwards | An association is assigned the wrong direction of cause and effect. | Could the outcome cause the apparent exposure, or could influence run both ways? |
| 6. Forgetting outside causes | A third factor is mistaken for a direct relationship between two variables. | What else could affect both? |
| 7. Believing a graph before reading it | Visual choices exaggerate, obscure, or reframe a difference. | What are the axes, units, denominator, and time window? |
These errors can enter at different stages. Sampling and measurement affect the data collected; analytical choices affect which results emerge; charts and headlines shape how results are understood; and decision-makers may treat uncertain estimates as exact facts. A reader cannot always repair a weak study just by inspecting its final number.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
1. Assuming small differences are meaningful
Suppose a poll reports that 52% of respondents support a proposal, compared with 50% in an earlier poll. That is a difference of 2 percentage points. Relative to 50%, it is a 4% increase, but that does not mean support rose by four percentage points. The distinction matters because headlines often switch between percentage points and percent change without making the change clear.
#1 Best Overall
Polls and studies estimate a population value from observations. Because samples vary, the reported estimate is not usually an exact measurement of the whole population. An uncertainty interval—such as a confidence interval—helps show which values are reasonably compatible with the data under the method’s assumptions. A margin of error is useful context for some surveys, but it is not a universal test for every comparison: the design, estimate, comparison, and analysis matter.
Before treating a small difference as a trend, ask how the data were collected, how many observations there were, whether the same method was used, and how wide the uncertainty is. If two estimates have overlapping error bars, that alone does not settle whether they differ; the bars might show standard deviations, standard errors, or confidence intervals, which mean different things. A formal comparison may be needed.
Also distinguish statistical from practical size. A two-point change could matter across a very large population or if concentrated among people facing serious consequences. In another context, it could be negligible. The number does not answer that value judgment by itself.
Free tools Windows power users keep installed
One-click scans. No signup required.
2. Confusing statistical significance with real-world importance
Statistical significance is a property of a result under a specified analysis and assumptions. It does not establish that a finding is important, unbiased, true, or causal. With a sufficiently large sample, a very small effect may be estimated precisely enough to pass a conventional significance threshold. With a small or noisy sample, an effect that could matter may remain uncertain.
That is why a headline such as “Treatment cuts risk by 50%” needs a baseline. If risk falls from 2 in 10,000 to 1 in 10,000, the relative reduction is 50%, but the absolute reduction is 1 in 10,000. If it falls from 20% to 10%, the same relative reduction corresponds to a 10-percentage-point absolute change. These are not equivalent decisions for patients, policymakers, or consumers.
Rank #2
Ask for the effect size, absolute as well as relative change, and an uncertainty interval. Consider the sample size, the relevant real-world threshold, and possible costs or harms. A p-value alone does not tell you the probability that a hypothesis is true or the chance that the result is “just luck.” And a result that is not statistically significant does not prove there is no effect; it may simply be too imprecise to distinguish among plausible possibilities.
3. Neglecting the distribution and the extremes
An average can conceal variation. If one group has a higher average score than another, the groups may still overlap substantially, and many people in the lower-average group may score above many in the higher-average group. The mean does not say how spread out the data are or how outcomes differ at the tails.
Look for the median, spread, percentiles, and distribution where they are relevant. A mean can be pulled by a small number of unusually high or low values; a median describes the middle observation. Neither is automatically the right summary. If a question concerns rare but serious outcomes, the tail may matter more than the typical case. If an intervention is evaluated for a broad population, an average effect may hide a subgroup that gains little or is harmed.
Claims about what happens at the extremes require care. A small shift in group averages can affect the proportion above a threshold, but the size and direction of that effect depend on the distribution, its variability, and how the threshold is defined. Not every dataset follows a bell-shaped, or normal, distribution.
Extreme observations also invite regression to the mean. If people are selected because their first measurement was unusually high, a later measurement will often be closer to their usual level, even without an intervention. That is one reason a before-and-after improvement in an unusually bad period is not, on its own, proof that a treatment or policy worked.
Rank #3
- Statistions, how to lie
- Darrell Huff
- Illustrated by Irving Genis
- New York - London 5 6 7 8 9 0
4. Trusting coincidence
Two lines can move together without one causing the other. The often-cited comparison between swimming-pool drownings and the number of films featuring Nicholas Cage is intentionally absurd: their apparent correlation is a reminder that a striking match in a dataset is not a causal explanation. See the example in the original Phys.org republication of the article.
There are many ways to find an apparently interesting pattern by chance. If analysts test enough outcomes, subgroups, time windows, or combinations of variables, some results may look unusual even when there is no stable underlying relationship. Searching first and inventing an explanation afterward is more vulnerable to false discoveries than testing a specific prediction chosen in advance. Selective reporting or repeated analysis without disclosure can make the evidence look stronger than it is.
Ask whether the pattern was predicted before the data were examined, how many other analyses were tried, whether the result was adjusted or interpreted in light of those comparisons, and whether it appears in independent data. Replication helps distinguish a durable finding from a one-off coincidence, though no single replication answers every question.
Correlation can still be useful. A variable may help predict an outcome without causing it. Prediction and causal explanation are different claims, and the evidence required for each is different.
5. Getting causation backwards
If A and B occur together, it is tempting to say A caused B. But the direction may be reversed, or influence may run both ways. Poor health can make it harder to keep a job, while unemployment can worsen health. Observing the association alone cannot tell you how much each direction contributes.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →Timing can help: a proposed cause generally needs to precede its effect. But temporal order is not enough to prove causation. An illness may lead clinicians to prescribe a treatment, so the treatment can appear associated with worse outcomes in an observational dataset because the sickest people received it. Likewise, police may be assigned to areas with high crime; the correlation between police presence and crime does not establish that police presence caused the crime.
To support a causal claim, researchers need a design and analysis that address plausible alternatives. Random assignment can, when ethical and feasible, make groups comparable on average and strengthen causal inference. Observational studies can also provide important evidence, but their conclusions depend on assumptions, measurement, and how well alternative explanations are addressed.
6. Forgetting outside causes
A confounder is a factor associated with both an apparent exposure and an outcome that can distort their observed relationship. Imagine a report that people who eat at restaurants more often have better cardiovascular health. Socioeconomic status might influence both how often someone eats out and their access to healthcare, working conditions, diet, or other health-related factors:
Socioeconomic status
↙ ↘
Restaurant meals Cardiovascular healthThe diagram is a simplified possibility, not proof that socioeconomic status explains the whole association. A confounder is something to investigate, not a universal escape hatch for dismissing results.
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteResearchers may adjust for measured confounders, but adjustment is not magic. It cannot reliably account for factors that were not measured or were measured poorly, and the result depends on the model and on which variables were included. “Control for everything” is not a safe rule: adjusting for a mediator (a step on the pathway from cause to outcome) can remove part of the effect a study aims to estimate, while conditioning on certain shared consequences can introduce selection or collider bias.
Best Value
A credible causal analysis starts with a clear question and a prespecified account of how variables may relate, then explains what was measured and why particular adjustments were made. Similar results under reasonable alternative analyses can add confidence, but they do not automatically eliminate unmeasured confounding.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.7. Believing the graph before reading it
A chart can use accurate data and still make a small difference look dramatic or a large one hard to see. A vertical axis that starts above zero can magnify visual differences between bars; a long time window can make a short-term change disappear. Neither choice is inherently dishonest: a clearly labeled truncated axis may be appropriate for displaying small changes. The question is whether the scale and design help readers judge magnitude fairly.
Check the labels, units, intervals, and denominator. Is the chart showing counts, percentages, rates, or cumulative totals? A rising cumulative total may simply reflect that values are being added over time; it is not the same as a rising rate. A percentage without the count behind it can hide whether it represents 5 of 10 people or 5,000 of 10,000.
Recommended Free Tools
Other visual choices can distort the message: unequal intervals, dual axes, three-dimensional bars, area or volume used to encode values, unlabeled logarithmic scales, color that overstates categories, hidden missing data, overlapping points, or smoothing that conceals volatility. A graph should identify its population, time period, denominator, and treatment of missing observations well enough for the comparison to be understood.
Related pitfalls beyond the seven
The seven-part framework is a starting point, not an exhaustive list. Several other problems often explain why a number or conclusion is unreliable:
- Selection and nonresponse bias: The people who enter a study or answer a survey may differ from those who do not. A large sample is not necessarily representative.
- Measurement error: A variable may be recorded inaccurately, defined differently across groups, or used as a poor proxy for the concept in the claim.
- Missing data: If missingness is related to outcomes or group membership, analyzing only complete records can skew the result.
- Base-rate neglect: A test’s sensitivity and specificity do not tell you, by themselves, how likely a positive result is to be correct. When the condition is rare, false positives can make up a substantial share of positive results.
- Relative-risk framing: A dramatic relative change can correspond to a small absolute change when the baseline risk is low.
- Cherry-picked outcomes or time periods: A result may look different when the start date, endpoint, subgroup, or outcome is changed. Ask why that comparison was chosen.
These issues can overlap. For example, an unrepresentative sample can produce a precise estimate for the wrong population; a selective analysis can find a coincidental pattern; and a graph can hide the denominator that would make the result easier to judge.
A five-minute check for a statistical claim
- What exactly was measured? Look for the population, variable definitions, and how data were collected.
- Compared with what? Identify the baseline, comparison group, time period, and whether groups are reasonably comparable.
- What is the denominator? Distinguish counts, rates, percentages, and cumulative totals; compare both absolute and relative changes.
- How uncertain is the estimate? Find the interval, sample size, and method. Remember that uncertainty depends on the design and analysis.
- Does the effect matter in practice? Consider the magnitude, who experiences it, and possible benefits and harms.
- Could there be another explanation? Consider coincidence, reverse causation, confounding, selection, and measurement problems.
- Does the chart show the comparison honestly? Check the axes, units, denominator, time window, and missing data.
- How many things were tested? A striking result is less persuasive if many outcomes or subgroups were searched without accounting for that.
- Does the conclusion apply to the people being discussed? A result in one sample or setting may not transfer to another population.
- Has it held up elsewhere? Independent replication and evidence from other appropriate designs can strengthen a claim.
The aim is not to distrust every statistic. It is to make the conclusion fit the evidence: a precise estimate can still be trivial, a useful predictor need not be a cause, and an uncertain result may justify more study rather than a confident yes or no.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
