Misusing Standard Error of the Mean (SEM)

Research conducted by: Peter Nagele

Case study prepared by: Robert F. Houser, Georgette Baghdady, and Jennifer E. Konick

Overview
Authors of published research articles often erroneously use the standard error of the mean to describe the variability of their study sample. Nagele demonstrated this misuse of the standard error of the mean as a descriptive statistic by manually searching four leading anesthesia journals in 2001.

Here are quotes on key points from Nagele’s article and our notes:

“Descriptive statistics aim to describe a given study sample without regard to the entire population.”

“If normally distributed, the study sample can be described entirely by two parameters: the mean and the standard deviation (SD).” However, a study sample variable is never exactly normally distributed. When a variable is close to normally distributed, the mean and median are quite similar. Therefore, the mean and SD would be sufficient.

“The SD represents the variability within the sample.” It tells us about “the distribution of individual data points around the mean.” The latter statement, however, is a generalization since the SD cannot tell us exactly where each data point lies relative to the mean.

“[I]nferential statistics generalize about a population on the basis of data from a sample of this population.”

The standard error of the mean (SEM) “is used in inferential statistics to give an estimate of how the mean of the sample is related to the mean of the underlying population.” It “informs us how precise our estimate of the [population] mean is.”

Thus, “the SEM estimates the precision and uncertainty [with which] the study sample represents the underlying population.”

The standard error of the mean is calculated by dividing the sample standard deviation by the square root of the sample size (SEM=SD/√n).

“[T]he SEM is always smaller than the SD.” However, this is only true as long as the sample size is greater than 1.

“In general, the use of the SEM should be limited to inferential statistics [for which] the author explicitly wants to inform the reader about the precision of the study, and how well the sample truly represents the entire population [of interest].” A sample never truly represents the population.

Questions to Answer
How prevalent is the inappropriate use of the SEM in describing the variability of the study sample in research publications? What is the proper use of the SEM?

Design Issues
The author focused on four leading anesthesia journals in his field of expertise. The misapplication of the SEM in descriptive statistics can be found in professional journals of many, if not all, fields of research.

Descriptions of Variables
Variable Description
Incorrect use of SEM; total

Total frequency of misuse of SEM; expressed as number of articles and percent
Laboratory studies using SEM incorrectly

A subset of the above variable; expressed as number of articles and percent
Correct use of SD

Frequency of correct use of standard deviation; expressed as number of articles and percent


References

Nagele, P. (2003). Misuse of standard error of the mean (SEM) when reporting variability of a sample. A critical evaluation of four anaesthesia journals. British Journal of Anaesthesia, 90, 514-516.

Hassani, H., Ghodsi, M., Howell, G. (2010). A note on standard deviation and standard error. Teaching Mathematics and Its Applications, 29, 108-112.

Altman, D. G., Bland, J. M. (2005). Standard deviations and standard errors. BMJ, 331, 903.

Links

Nagele article


Exercises

Please read the Nagele article before performing the exercises.

  1. What is the most appropriate way to portray graphically the results shown in Table 1? Design and create the graphs and show them to a few naïve individuals (you can use your friends) to see if they can interpret the graphs correctly.
  2. Look over recent editions of your favorite journal and search for at least two articles that misuse the standard error of the mean. Also, see if you can spot any articles that properly report the standard error of the mean.