Factorial Design

A factorial design is an experimental design in which two or more independent variables (called factors) are manipulated simultaneously, allowing researchers to examine both the individual effects of each factor (main effects) and the combined effects of factors acting together (interaction effects). In a full factorial design, every possible combination of factor levels is tested, creating a fully crossed design.

Most real world phenomena are influenced by multiple variables acting in combination. Factorial designs allow researchers to study these combinations efficiently. Instead of running separate experiments for each factor, a factorial design tests all factors at once and provides information about interactions that separate single factor experiments cannot detect. When the factors do not interact, it can also reduce the total number of participants needed, because each main effect is estimated from everyone in the study. Interaction effects are often the most interesting findings; for example, a medication might work well in patients who also receive behavioral therapy but not in those who do not, which is a medication by therapy interaction. Testing one factor at a time would miss such interactions and might lead to incorrect conclusions about the effectiveness of interventions across different conditions.

A factorial design is described by the number of factors and their levels. A 2 x 2 design has two factors, each with two levels, creating four experimental conditions. A 2 x 3 design has two factors, one with two levels and one with three levels, creating six conditions. A 2 x 2 x 3 design has three factors, creating twelve conditions.

Participants are randomly assigned to one of the conditions (in a between subjects design) or experience all conditions (in a within subjects design). Continuous outcomes are commonly analyzed using factorial ANOVA (or an equivalent regression model), which partitions the total variance into main effects for each factor, interaction effects for each combination of factors, and residual (error) variance.

A main effect is the overall effect of one factor, averaged across the levels of the other factor(s). An interaction effect occurs when the effect of one factor depends on the level of another factor. Interaction effects can be ordinal (the direction of the effect is the same across levels, but the magnitude differs) or disordinal (the direction reverses). Disordinal interactions are particularly important because they mean that general statements about the effect of one factor are misleading without specifying the level of the other factor.

In a 2 x 2 trial, each main effect can be estimated at the margins of the table: everyone who received one level of factor A is compared with everyone who received the other, whatever they received of factor B. Because all participants contribute to both comparisons, one factorial trial can answer two questions with roughly the sample that one of them would need. That efficiency rests on the assumption that the factors do not interact. If they do, the margin comparison averages two different effects, and the results have to be read inside the table, comparing the individual cells (for example, A alone against neither treatment), each of which holds only a quarter of the participants. Detecting an interaction as large as the main effects, with the same power, needs about four times the sample size.

As the number of factors increases, the number of conditions multiplies (a 2 x 2 x 2 x 2 design has sixteen), so each cell holds fewer participants and interactions, especially higher order ones, are estimated with less power. Fractional factorial designs, in which only a subset of all possible combinations is tested, can address this problem when higher order interactions are expected to be negligible.

Factorial designs are appropriate when the research question involves the effects of multiple independent variables and their potential interactions. They are used in pharmaceutical research (testing drug versus placebo crossed with behavioral therapy versus no therapy), agricultural research (testing fertilizer type crossed with irrigation level), educational research (testing teaching method crossed with class size), and industrial research (testing temperature crossed with pressure in manufacturing). They are the design to use when the interaction between factors is itself a research question, provided the study is large enough to estimate it; their efficiency in answering several questions at once depends on the factors not interacting.

A common error is ignoring interaction effects and interpreting main effects alone. If a significant interaction is present, main effects can be misleading because the effect of one factor differs across levels of the other. Conversely, a non significant interaction test does not show that the factors do not interact, because factorial studies are often too small to detect interactions. Another mistake is creating overly complex designs with too many factors, resulting in cells with very few participants and inadequate statistical power. Researchers sometimes fail to ensure equal or proportional sample sizes across cells; with unequal cells the factors are no longer independent of each other, so the main effect tests depend on how the sums of squares are calculated (Type I, II or III) and need careful interpretation. Finally, neglecting to report simple effects analysis when an interaction is significant leaves the reader unable to understand the nature of the interaction.

A clinical researcher uses a 2 x 2 factorial design to test the effects of exercise type (aerobic vs. resistance) and dietary supplement (protein supplement vs. placebo) on muscle mass gain in 200 older adults over 16 weeks, with 50 participants randomized to each of the four combinations. The mean gains are 1.8 kg with resistance exercise and supplement, 0.6 kg with resistance exercise and placebo, 0.4 kg with aerobic exercise and supplement, and 0.3 kg with aerobic exercise and placebo, with a standard deviation of about 1.6 kg in each group. The ANOVA shows a significant main effect of exercise type, F(1, 196) = 14.11, p < 0.001, a significant main effect of supplement, F(1, 196) = 8.25, p = 0.005, and a significant interaction, F(1, 196) = 5.91, p = 0.016. At the margins, the supplement adds 0.65 kg on average, but that average describes neither exercise group: simple effects analysis shows that the supplement adds 1.2 kg with resistance exercise, t(196) = 3.75, p < 0.001, and only 0.1 kg with aerobic exercise, t(196) = 0.31, p = 0.75.