Misconceptions
Prerequisites
Introduction
to Hypothesis Testing, Statistical
Significance, Type I and II Errors
Learning Objectives
 State why the probability value is not the probability the null hypothesis
is false
 Explain why a low probability value does not necessarily mean there
is a large effect
 Explain why a nonsignificant outcome does not mean the null hypothesis
is probably true
Misconceptions about significance testing are
common. This section lists three important ones.
 Misconception: The probability value is the probability that
the null hypothesis is false.
Proper interpretation: The probability value is the probability
of a result as extreme or more extreme given that the null hypothesis
is true.
It is the probability of the data given the null hypothesis. It is not the probability
that the null hypothesis is false.
 Misconception: A low probability
value indicates a large effect.
Proper interpretation: A low probability value indicates that
the sample outcome (or one more extreme) would be very unlikely
if the null hypothesis were true. A low probability value can
occur with small effect sizes, particularly if the sample size
is large.
 Misconception: A
nonsignificant outcome means that the null hypothesis is probably
true.
Proper interpretation:
A nonsignificant outcome means that the data do not conclusively demonstrate
that the null hypothesis is false.
