Statistics that matter: p-values, risk and effect size
The handful of statistical ideas you need to judge a study, explained with plain examples.
VALREXA Biosciences · 8 min read · November 2026
- A p-value tells you how surprising the data would be if there were no real effect, not whether a result matters.
- Confidence intervals show the range of plausible effects.
- Relative risk can make small effects sound large; always look for absolute risk.
The p-value
A p-value is the probability of seeing results at least this extreme if there were truly no effect. By convention, p < 0.05 is called “statistically significant.” It does not mean there is a 95% chance the finding is true, and it says nothing about whether the effect is large enough to matter.
Confidence intervals
A 95% confidence interval gives a range of effect sizes consistent with the data. A narrow interval means a precise estimate. If the interval for a difference includes zero (or 1.0 for a ratio), the result is compatible with no effect.
Statistical versus clinical significance
With enough participants, a tiny, meaningless difference can be statistically significant. Always ask whether the size of the effect would matter to a patient.
Relative versus absolute risk
Example: A treatment lowers the risk of an event from 2 in 100 people to 1 in 100.
Relative risk reduction: 50% (“cuts risk in half!”).
Absolute risk reduction: 1 percentage point.
Number needed to treat (NNT): 1 ÷ 0.01 = 100 people treated to prevent one event.
Both numbers are true, but they leave very different impressions. Good reporting gives both.
Other traps
| Trap | What it means |
|---|---|
| Small samples | More likely to produce false or exaggerated findings |
| Multiple comparisons | Test 20 outcomes and one will likely hit p < 0.05 by chance |
| Regression to the mean | Extreme measurements tend to move toward average on their own |
| Placebo effect | People often feel better from expectation alone, which is why controls matter |
| Survivorship bias | Looking only at successes, such as people who stayed in a study |
| Surrogate endpoints | A lab marker improves but real outcomes may not |
Practice these ideas with our worked practice problems.
Further reading
These are authoritative background resources rather than citations to a specific experimental finding. This guide is educational, not medical advice, and does not endorse the use of experimental compounds.