Interpreting Significant Results
Prerequisites
Introduction
to Hypothesis Testing, Statistical
Significance, Type I and II Errors,
One and TwoTailed Tests
Learning Objectives
 Discuss whether rejection of the null hypothesis should be an allornone
proposition
 State the value of a significance test when it is extremely likely
that the null hypothesis of no difference is false even before doing
the experiment.
When a probability
value is below the α level, the effect is statistically
significant and the null hypothesis is rejected.
However, not all statistically significant effects should be treated
the same way. For example, you should have less confidence that
the null hypothesis is false if p = 0.049 than p = 0.003. Thus,
rejecting the null hypothesis is not an allornone proposition.
If the null hypothesis is rejected, then the alternative
to the null hypothesis (called the alternative
hypothesis) is accepted.
There are many situations in which it is very unlikely
two conditions will have exactly the same population means. For
example, it is practically impossible that aspirin and acetaminophen
provide exactly the same degree of pain relief. Therefore, even
before an experiment comparing their effectiveness is conducted,
the researcher knows that the null hypothesis of exactly no difference
is false. However, the researcher does not know which drug offers
more relief. If a test of the difference is significant, then
the direction of the difference is established. This point is
also made in the section on the relationship between confidence
intervals and significance tests.
