All Pairwise Comparisons Among Means
Between Two Means (Independent Groups)
- Define pairwise comparison
- Describe the problem with doing t tests among
all pairs of means
- Calculate the Tukey hsd test
- Explain why Tukey test should not necessarily be considered a follow-up
Many experiments are designed to compare more
than two conditions. As the number of tests increases, the probability
that one or more of the tests will result in a Type I error increases.
The Type I error rate can be controlled using a
test called the Tukey Honestly Significant Difference test or
Tukey HSD for short. The Tukey HSD is
based on a variation of the t distribution
that takes into account the number of means being compared. This
distribution is called the studentized range
The Tukey HSD Test is computed as follows:
- Compute the means and variances of each group.
- Compute MSE which is simply the mean of the variances.
for each pair of means where Mi is one mean, Mj is
the other mean, and n is the number of scores in each group.
- Compute p for each comparison using the Studentized
Range Calculator. The degrees of freedom is equal to
the total number of observations minus the number of means.
The assumptions of the Tukey test are essentially
the same as for an independent-groups
t test: normality, homogeneity of variance, and independent
observations. The test is quite robust to violations of normality.
Violating homogeneity of variance can be more problematical than
in the two-sample case since the MSE is based on data from all
groups. The assumption of independence of observations is important
and should not be violated.