A Comparison of Methods to Test for Mediation in Multisite Experiments
Multivariate Behavioral ResearchPublished 1 January 2005
Keenan A. Pituch, Tiffany A. Whittaker, Laura M. Stapleton
Citations109
SJR quartileQ1
SJR score2.71
SNIP2.12
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A Monte Carlo study extended the research of MacKinnon, Lockwood, Hoffman, West, and Sheets (2002) for single-level designs by examining the statistical performance of four methods to test for mediation in a multilevel experimental design, providing new evidence of the benefits of and further support for using the asymmetric confidence limits approach to Test for mediation.
Abstract
A Monte Carlo study extended the research of MacKinnon, Lockwood, Hoffman, West, and Sheets (2002) for single-level designs by examining the statistical performance of four methods to test for mediation in a multilevel experimental design. The design studied was a two-group experiment that was replicated across several sites, included a single intervening variable and outcome, and assumed that the effects of the treatment and mediator were constant across sites. The findings provide new evidence of the benefits of and further support for using the asymmetric confidence limits approach to test for mediation. In addition, the authors provide further support for using confidence intervals to assess if treatment effects are completely mediated, as using traditional hypothesis testing may lead to erroneous conclusions.
Keywords
PsychologySocial Sciences
Journal of Personality and Social PsychologyThe moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations.
72,132 Citations1986Reuben M. Baron, David A. Kenny
Sociological MethodologyAsymptotic Confidence Intervals for Indirect Effects in Structural Equation Models
12,501 Citations1982Michael E. Sobel
ScienceNeighborhoods and Violent Crime: A Multilevel Study of Collective Efficacy
11,862 Citations1997Robert J. Sampson, Stephen W. Raudenbush +1 more
Multilevel analyses showed that a measure of collective efficacy yields a high between-neighborhood reliability and is negatively associated with variations in violence, when individual-level characteristics, measurement error, and prior violence are controlled.
Psychological MethodsMediation in experimental and nonexperimental studies: New procedures and recommendations.
10,909 Citations2002Patrick E. Shrout, Niall Bolger
The authors recommend that with small to moderate samples, bootstrap methods be used to assess mediation and argue that R. Kenny's (1986) recommendation of first testing the X --> Y association for statistical significance should not be a requirement when there is a priori belief that the effect size is small or suppression is a possibility.
Psychological MethodsA comparison of methods to test mediation and other intervening variable effects.
9,145 Citations2002David P. MacKinnon, Chondra M. Lockwood +3 more
A Monte Carlo study compared 14 methods to test the statistical significance of the intervening variable effect and found two methods based on the distribution of the product and 2 difference-in-coefficients methods have the most accurate Type I error rates and greatest statistical power.
Multivariate Behavioral ResearchConfidence Limits for the Indirect Effect: Distribution of the Product and Resampling Methods
7,629 Citations2004David P. MacKinnon, Chondra M. Lockwood +1 more
Two alternatives for improving the performance of confidence limits for the indirect effect are evaluated: a method based on the distribution of the product of two normal random variables, and resampling methods.
Prevention ScienceEquivalence of the Mediation, Confounding and Suppression Effect
4,037 Citations2000David P. MacKinnon, Jennifer L. Krull +1 more
The statistical similarities among mediation, confounding, and suppression are described and methods to determine the confidence intervals for confounding and suppression effects are proposed based on methods developed for mediated effects.
Multilevel Analysis
2,391 Citations2002Joop J. Hox, Mirjam Moerbeek +1 more
National Academies Press eBooksScientific Research in Education
1,735 Citations2002Richard J. Shavelson
Multivariate Behavioral ResearchA Simulation Study of Mediated Effect Measures
1,698 Citations1995David P. MacKinnon, Ghulam Warsi +1 more
Analytical solutions for point and variance estimators of themediated effect, the ratio of the mediated to the direct effect, and the proportion of the total effect that is mediated were studied with statistical simulations.
Multivariate Behavioral ResearchMultilevel Modeling of Individual and Group Level Mediated Effects
1,311 Citations2001Jennifer L. Krull, David P. MacKinnon
Multilevel mediational modeling methods were also applied to data from a preventive intervention designed to reduce intentions to use steroids among players on high school football teams and shows how the multilevel technique may lead to more accurate results.
Psychological MethodsMediation in experimental and nonexperimental studies: New procedures and recommendations.
1,026 Citations2002Patrick E. Shrout, Niall Bolger
Psychological MethodsLower level mediation in multilevel models.
689 Citations2003David A. Kenny, Josephine D. Korchmaros +1 more
Sociological Methods & ResearchDirect and Indirect Effects in Linear Structural Equation Models
682 Citations1987Michael E. Sobel
Psychological MethodsStatistical power and optimal design for multisite randomized trials.
581 Citations2000Stephen W. Raudenbush, Xiaofeng Liu
A standardized hierarchical linear model is proposed and rules of thumb similar to those proposed by J. Cohen (1988) for small, medium, and large effect sizes and for small and large treatment-by-site variance are used.
American Educational Research JournalAnswers and Questions About Class Size: A Statewide Experiment
528 Citations1990Jeremy D. Finn, Charles M. Achilles
The Future of ChildrenThe Tennessee Study of Class Size in the Early School Grades
520 Citations1995Frederick Mosteller
The Tennessee class size project is a three-phase study designed to determine the effect of smaller class size in the earliest grades on short-term and long-term pupil performance and it was clear that smaller classes did produce substantial improvement in early learning and cognitive studies.
Journal of Educational StatisticsStandard Errors and Sample Sizes for Two-Level Research
457 Citations1993Tom A. B. Snijders, Roel Bosker
Structural Equation Modeling A Multidisciplinary JournalInvestigation of Mediational Processes Using Parallel Process Latent Growth Curve Modeling
422 Citations2003JeeWon Cheong, David P. MacKinnon +1 more
This study investigated a method to evaluate mediational processes using latent growth curve modeling, and empirical data from a longitudinal drug prevention program, Adolescents Training and Learning to Avoid Steroids, were used.
Evaluation ReviewMultilevel Mediation Modeling in Group-Based Intervention Studies
414 Citations1999Jennifer L. Krull, David P. MacKinnon
These methods were applied to data from an existing substance use intervention to show the impact multilevel mediation modeling can have on the conclusions drawn from real-world evaluation studies.
Journal of Educational and Behavioral StatisticsDesign Issues for Experiments in Multilevel Populations
146 Citations2000Mirjam Moerbeek, Gerard van Breukelen +1 more
Prevention ScienceMediating Mechanisms in a Program to Reduce Intentions to Use Anabolic Steroids and Improve Exercise Self-Efficacy and Dietary Behavior
138 Citations2001David P. MacKinnon, Linn Goldberg +6 more
Although the results differed across the three dependent variables, the program appeared to work by changing team norms and changes in knowledge and perceived severity were mediators of program effects in this study.
Journal of the Royal Statistical Society Series B (Statistical Methodology)On the Distribution of Products of Random Variables
110 Citations1967Z. A. Łomnicki
BJOG An International Journal of Obstetrics & GynaecologyCotinine‐assisted intervention in pregnancy to reduce smoking and low birthweight delivery
74 Citations1991James E. Haddow, George J. Knight +3 more
This study aims to investigate the feasibility and impact of integrating a cotinine‐assisted smoking intervention programme with an existing antenatal maternal serum alpha‐fetoprotcin screening service for open neural tube defects.
Educational Research and EvaluationA Multilevel Perspective on the Design and Analysis of Intervention Studies
53 Citations1998Ian Plewis, Jane Hurry
TechnometricsSelected Tables in Mathematical Statistics (Vol. VII), The Product of Two Normally Distributed Random Variables
51 Citations1983Melvin D. Springer
Evaluation ReviewStudying Variation in Program Success
39 Citations1994Michael Seltzer
It is shown how multilevel modeling techniques provide a means of moving beyond estimating overall, average program effects to investigations of how differences in various aspects of implementation across sites relate to differences in program success.
Elsevier eBooksAssessing the Impact of a Decentralised Initiative: the British Technical and Vocational Education Initiative
13 Citations1991David Raffe
The Journal of Experimental EducationUsing Multilevel Modeling in Large-Scale Planned Variation Educational Experiments: Improving Understanding of Intervention Effects
11 Citations2001Keenan A. Pituch
The author used data based on a large-scale educational field experiment to describe multilevel models that one can use to address a set of research questions: how an educational intervention produces its results, for whom the Intervention works, and whether the intervention works well across different contexts.
Educational Research and EvaluationStrengthening Multisite Educational Interventions: An Illustration With Multilevel Modeling
6 Citations1999Keenan A. Pituch, John W. Miller
The value and use of multilevel models to examine site-to-site outcome differences in the analysis of data from multisite evaluations are illustrated and how program developers can exploit these differences in an attempt to understand and strengthen an intervention is shown.