|
Example Calculations
Prerequisites
Introduction
to Power, Binomial
Distribution,
Testing a Single Mean
Learning Objectives
- Compute power using the binomial distribution
- Compute power using the normal distribution
- Use a power calculator to compute power for the t distribution
In the "Shaking
and Stirring Martinis" case study, the question was whether
Mr. Bond could tell the difference between martinis that were stirred
and martinis that were shaken. For the sake of this example, assume
he can tell the difference and is able to correctly state whether
a martini had been shaken or stirred 0.75 of the time. Now, suppose
an experiment is being conducted to investigate whether Mr. Bond
can tell the difference. Specifically is Mr. Bond correct more
than 0.50 of the time. We know that he can (that's an assumption
of the example). However, the experimenter does not know and asks
Mr. Bond to judge 16 Martinis. The experimenter will do a significance test based on the binomial
distribution. Specifically, if a one
tailed test is significant at the 0.05 level then
he or she will conclude that Mr. Bond can tell the difference.
The probability value is computed assuming the null hypothesis is true
(π = 0.50). Therefore,
the experimenter will determine how many times Mr. Bond is correct,
and compute the probability of being correct that many or more
times given that the null hypothesis is true. The question is:
what is the probability the experimenter will correctly reject
the null hypothesis that π = 0.50? In other
words, what is the power of this experiment.
The binomial distribution for N = 16 and π =
0.50 is shown in Figure 1. The probability of being correct
on 11 or more trials is 0.105 and the probability of being correct
on 12 or more trials is 0.038. Therefore, the probability of being
correct on 12 or more trials is less than 0.05. This means that
the null hypothesis will be rejected if Mr. Bond is correct on
12 or more trials and will not be rejected otherwise.
We know that Mr. Bond is correct 0.75 of the time. (Obviously
the experimenter does not know this or there would be no need
for an experiment.) The binomial distribution with N = 16 and π =
0.75 is shown in Figure 2.
The probability of being correct on 12 or
more trials is 0.63. Therefore, the power
of the experiment is 0.63.
To sum up, the probability of being correct on 12
or more trials given that the null hypothesis is true is less
than 0.05. Therefore, if Mr. Bond is correct on 12 or more trials,
the null hypothesis will be rejected. Given Mr. Bond's true ability
to be correct on 0.75 of the trials, the probability he will be
correct on 12 or more trials is 0.63. Therefore power is 0.63.
In
the section on Testing
a Single Mean for significance, the first example was based on the assumption
that the experimenter knew the population variance. Although
this is rarely true in practice, the example is very useful
for pedagogical purposes. For the same reason, the following
example assumes the experimenter knows the population variance.
Power
calculators are available for situations in which the
experimenter does not know the population variance.
Suppose a math achievement test were known to have
a mean of 75 and standard deviation of 10. A researcher is interested
in whether a new method of teaching results in a higher mean.
Assume that although the experimenter does not know it, the population
mean is 80. The researcher plans to sample 25 subjects and do
a one-tailed test of the whether the sample mean is significantly
higher than 75. What is the probability that the researcher will
correctly reject the false null hypothesis that the population
mean is 75? The following shows how this probability is computed.
The researcher assumes that the population standard
deviation with the new method is the same as with the old method
(10) and that the distribution is normal. Since the population
standard deviation is assumed to be known, the researcher can
use the normal distribution rather
than the t
distribution to
compute the p value. Recall that the standard error of the mean
(σM)
is
which is equal to 10/5 = 2 in this example. As
can be seen in Figure 3, if the null hypothesis that the population
mean equals 75 is true, then the probability of a sample mean
being greater than or equal to 78.29 is 0.05. Therefore, the experimenter
will reject the null hypothesis if the sample mean, M, is 78.29
or larger.
The question, then, is what is the probability
the experimenter gets a sample mean greater than 78.29 given that
the population mean is 80? Figure 4 shows that this probability
is 0.80.
Therefore, the probability that the experimenter
will reject the null hypothesis that the null hypothesis is 75
is 0.80. In other words, power = 0.80.
Calculation of power is more complex for t-tests
and for Analysis of Variance. The power calculator computes power
for a t test of independent groups. Calculators for other types
of designs can be found at this
Power Calculator for Independent t test
Other
Power Calculators (external link)
|