Correcting Effect Sizes for Score Reliability: A Reminder that Measurement and Substantive Issues are Linked Inextricably
Educational and Psychological MeasurementPublished 1 April 2002
Frank G. Baugh
Citations86
SJR quartileQ1
SJR score1.93
SNIP2.21
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
Abstract
The present article emphasizes that measurement issues must be explicitly considered even in studies that focus on substantive questions. First, dynamics associated with insufficient attention being paid to score reliabilities in substantive studies are discussed. Next, reasons to adjust effect size indices for score unreliability are presented. Finally, some procedures for adjusting effect sizes for score reliability are briefly reviewed.
Keywords
Decision SciencesEconomics, Econometrics and Finance
Computers Environment and Urban SystemsStatistical power analysis for the behavioral sciences
65,645 Citations1990
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings
5,538 Citations2015Frank L. Schmidt, John E. Hunter
Journal of the Royal Statistical Society Series D (The Statistician)Meta-Analytic Procedures for Social Research.
4,283 Citations1994Andrew Metcalfe, R. Rosenthal
Journal of the American Statistical AssociationMeasurement, Design, and Analysis: An Integrated Approach.
4,066 Citations1992Sandra G. Funk, Elazar J. Pedhazur +1 more
Occupational Therapy In Health CarePublication Manual of the American Psychological Association
3,198 Citations2011Kate E. DeCleene, Jennifer Fogo
American PsychologistStatistical methods in psychology journals: Guidelines and explanations.
2,773 Citations1999Leland Wilkinson, Task Force on Statistical Inference
Educational and Psychological MeasurementPractical Significance: A Concept Whose Time Has Come
1,792 Citations1996Roger E. Kirk
Academy of Management ReviewMeta-Analysis: Cumulating Research Findings across Studies
1,216 Citations1984Jack M. Feldman, John E. Hunter +2 more
BMJThe Handbook of Research Synthesis
1,061 Citations1994R Hanka
Meta- analysis, as the statistical analysis of a large collection of results from individual studies is called, has now achieved a status of respectability in medicine.
Measurement and Evaluation in Counseling and DevelopmentUnderstanding Internal Consistency Reliability Estimates: A Conceptual Primer on Coefficient Alpha
1,057 Citations2001Robin K. Henson
Journal of Wildlife ManagementThe Journal of Wildlife Management
709 Citations2011Scott Mcnay, Joan Voller
The results suggest that conservation of intact sagebrush vegetation communities could benefit ≥14 reptiles, a greater number than previously estimated, and additional research on each species' response to habitat restoration actions is needed to assess broader claims of multi‐ taxa benefits when it comes to manipulative sage‐grouse habitat management.
Educational and Psychological MeasurementReliability Generalization: Exploring Variance in Measurement Error Affecting Score Reliability Across Studies
446 Citations1998Tammi Vacha‐Haase
Journal of Counseling & Development“Statistical,” “Practical,” and “Clinical”: How Many Kinds of Significance Do Counselors Need to Consider?
400 Citations2002Bruce Thompson
Journal of Business ResearchMeta-analytic procedures for social research
349 Citations1994Elizabeth J. Wilson
Educational ResearcherAERA Editorial Policies regarding Statistical Significance Testing: Three Suggested Reforms
343 Citations1996Bruce Thompson
Educational and Psychological MeasurementPsychometrics is Datametrics: the Test is not Reliable
342 Citations2000Bruce Thompson, Tammi Vacha‐Haase
Psychological BulletinMagnitude of experimental effect and a table for its rapid estimation.
317 Citations1968H. A. FRIEDMAN
The Journal of Experimental EducationEvaluating Results Using Corrected and Uncorrected Effect Size Estimates
265 Citations1993Patricia Snyder, Stephen Lawson
American PsychologistGraduate training in statistics, methodology, and measurement in psychology: A survey of PhD programs in North America.
238 Citations1990Leona S. Aiken, Stephen G. West +6 more
Educational and Psychological Measurement
235 Citations2018W. Holmes Finch, Brian F. French
Psychological MethodsThe appropriate use of null hypothesis testing.
213 Citations1996Robert W. Frick
The Journal of Experimental EducationSignificance, Effect Sizes, Stepwise Methods, and Other Issues: Strong Arguments Move the Field
212 Citations2001Bruce Thompson
Educational ResearcherResearch news and Comment: AERA Editorial Policies Regarding Statistical Significance Testing: Three Suggested Reforms
210 Citations1996Bruce Thompson
The logic of statistical significance testing is presented in an accessible manner; many people who use statistical tests might not place such a premium on the tests if these individuals understood what the tests really do, and what they do not do.
Educational and Psychological MeasurementSample Compositions and Variabilities in Published Studies versus Those in Test Manuals: Validity of Score Reliability Inductions
187 Citations2000Tammi Vacha‐Haase, Lori R. Kogan +1 more
The Journal of Experimental EducationThe Use of Statistical Significance Tests in Research
184 Citations1993Bruce Thompson
Psychological MethodsOn the logic and purpose of significance testing.
172 Citations1997José M. Cortina, William P. Dunlap
Theory & PsychologyIf Statistical Significance Tests are Broken/Misused, What Practices Should Supplement or Replace Them?
120 Citations1999Bruce Thompson
Educational ResearcherResearch Techniques in <i>AERJ</i> Articles: 1969 to 1978
103 Citations1980Victor L. Willson
The Journal of Experimental EducationPractices Regarding Reporting of Reliability Coefficients: A Review of Three Journals
97 Citations1999Tammi Vacha‐Haase, Carin M. Ness +2 more
Journal of Wildlife ManagementOn the Past and Future of Null Hypothesis Significance Testing
90 Citations2002Daniel H. Robinson, Howard Wainer
It is suggested that the historical use of null hypothesis significance testing was reasonable and that current users might spend time profitably reading some of Fisher's applied work, but modifications to NHST, and to the interpretations of its outcomes, might better suit the needs of modern science.
Journal of Counseling PsychologyTrends in reporting psychometric properties of scales used in counseling psychology research.
78 Citations1990Scott T. Meier, Susan R. Davis
Journal of research and development in educationState of the art in statistical significance and effect size reporting: A review of the APA Task Force report and current trends.
71 Citations2000Robin K. Henson, A. Delany Smith
Educational and Psychological MeasurementHow Well Do Researchers Report their Measures? an Evaluation of Measurementin Published Educational Research
63 Citations1998Dale Whhtington
Blackwell Publishing Ltd eBooksHandbook of Research Methods in Experimental Psychology
59 Citations2005
Educational and Psychological MeasurementAlphamax: A Program that Maximizes Coefficient Alpha by Selective Item Deletion
24 Citations1990Bruce Thompson
Educational and Psychological MeasurementReporting Effect Sizes: the Roles of Editors, Textbook Authors, and Publication Manuals
20 Citations2001Janet Shibley Hyde