Differences between Two Means (Independent Groups)

Prerequisites
Sampling Distribution of Difference between Means, Confidence Intervals, Confidence Interval on the Difference between Means, Hypothesis Testing, Testing a Single Mean

Learning Objectives

  1. State the assumptions for testing the difference between two means
  2. Estimate the population variance assuming homogeneity of variance
  3. Compute the standard error of the difference between means
  4. Compute t and p for the difference between means.

It is much more common for a researcher to be interested in the difference between means than in the specific values of the means themselves. This section covers how to test for differences between means from two separate groups of subjects. A later section describes how to test for differences between the means of two condition in designs in where only one group of subjects is used and each subject is tested in each condition.

In order to test whether there is a difference between population means, we are going to make three assumptions:

    1. The two populations have the same variance. This assumption is called the assumption of homogeneity of variance.
    2. The populations are normally distributed.
    3. Each value is sampled independently from each other value. This assumption requires that each subject provide only one value. If a subject provides two scores, then the value are not independent. The analysis of data with two scores per subject is shown in the section on the correlated t test later in this chapter.

The consequences of violating the former two assumptions are investigated in the simulation in the next section. For now, suffice it to say that small-to-moderate violations of assumptions 1 and 2 do not make much difference. It is important not to violate assumption 3.

We saw the following general formula for significance testing in the section on testing a single mean:

The first step is to compute the statistic, which is simply the difference between means.

The next step is to compute the estimate of the standard error of the statistic. In this case, the statistic is the difference between means so the estimated standard error of the statistic is (). Recall from the relevant section in the chapter on sampling distributions that the formula for the standard error of the difference in means in the population is:


In order to estimate this quantity, we estimate σ2 and use that estimate in place of σ2. Since we are assuming the population variances are the same, we estimate this variance by averaging our two sample variances. Thus, our estimate of variance is computed using the following formula:

 

The next step is to compute t by plugging these values into the formula.

Finally, we compute the probability of getting a t as large or larger than the computed t or as small or smaller than that. To do this, we need to know the degrees of freedom. The degrees of freedom is the number of independent estimates of variance on which MSE is based. This is equal to (n1 -1) + (n2 -1) where n1 is the sample size for the first group and n2 is the sample size of the second group.

Once we have the degrees of freedom, we can use the t distribution calculator to find the probability.