Effect Size Statistics: An Analysis of Statistics Textbooks Used in Psychology and Education.

Abstract

The purpose of this research was to analyze recent statistics textbooks (n.22) to examine the ways in which authors addressed the issue of effect size (ES) and the practical significance of research results. In terms of the overall prevalence of ES statistics, the one-sample, matched-pair, and two-sample t-tests, and one-factor analysis of variance were presented in all 22 texts. ES statistics corresponding to each of these significance tests were presented in 4, 9, 16, and 18 texts, respectively. Problems with the way in which ES statistics were presented were identified, including the failure to distinguish between ES parameters and statistics, the use of conceptually uninformative cookbook formulas, and the lack of agreement on how to calculate specific ES statistics. Problems with the ways in which authors discussed the interpretation of the magnitude of the ES statistics were also identified, including the over-reliance on of (Contains 3 tables and 66 references.) (Author/SLD) Reproductions supplied by EDRS are the best that can be made from the original document. PERMISSION TO REPRODUCE AND DISSEMINATE THIS MATERIAL HAS BEEN GRANTED BY cv,t,s TO THE EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) 1 U.S. DEPARTMENT OF EDUCATION Office of Educational Research and Improvement ED ATIONAL RESOURCES INFORMATION CENTER (ERIC) This document has been reproduced as received from the person or organization originating it. 0 Minor changes have been made to improve reproduction quality. Points of view or opinions stated in this document do not necessarily represent official OERI position or policy. Effect Size Statistics: An Analysis of Statistics Textbooks used in Psychology and Education Deborah A. Curtis Cheri J. Araki San Francisco State University Paper presented at the April, 2002 American Educational Research Association, New Orleans, LA. 2 BEST COPY AVAILABLE Evaluating Statistics Texts 1 Abstract The purpose of this research was to analyze recent statistics textbooks (N=22) to examine the ways in which authors addressed the issue of effect size (ES) and the practical significance of research results. In terms of the overall prevalence of ES statistics, the one-sample, matchedpair, and two-sample t-tests, and one-factor ANOVA were presented in all 22 texts. ES statistics corresponding to each of these significance tests were presented in 4, 9, 16, and 18 texts, respectively. Problems with the way in which ES statistics were presented were identified, including the failure to distinguish between ES parameters and statistics, the use of conceptually uninformative cookbook formulas, and the lack of agreement on how to calculate specific ES statistics. Problems with the ways in which authors discussed the interpretation of the magnitude of the ES statistics were also identified, including the over-reliance on rules of thumb.The purpose of this research was to analyze recent statistics textbooks (N=22) to examine the ways in which authors addressed the issue of effect size (ES) and the practical significance of research results. In terms of the overall prevalence of ES statistics, the one-sample, matchedpair, and two-sample t-tests, and one-factor ANOVA were presented in all 22 texts. ES statistics corresponding to each of these significance tests were presented in 4, 9, 16, and 18 texts, respectively. Problems with the way in which ES statistics were presented were identified, including the failure to distinguish between ES parameters and statistics, the use of conceptually uninformative cookbook formulas, and the lack of agreement on how to calculate specific ES statistics. Problems with the ways in which authors discussed the interpretation of the magnitude of the ES statistics were also identified, including the over-reliance on rules of thumb.