Multivariate Analysis

Multivariate analysis encompasses a set of statistical methods designed to analyze data involving several variables jointly, typically more than one dependent variable, accounting for the relationships and correlations among them. It is distinct from multivariable analysis, which involves multiple independent variables but only a single outcome.

Many research scenarios involve several outcomes that are naturally correlated. A clinical trial may measure both pain scores and functional ability, which tend to move together. A psychological study may assess multiple dimensions of wellbeing simultaneously. Analyzing each outcome separately (one at a time) ignores the correlations among them, inflates the number of statistical tests, and can miss patterns that are apparent only when outcomes are considered jointly. Multivariate methods address these issues by modeling the outcomes together, which can give a more complete picture of the relationships in the data.

Multivariate methods extend univariate techniques to analyze several variables jointly. Some, such as MANOVA and multivariate regression, model a vector of outcomes; others, such as PCA and factor analysis, analyze a set of variables without dividing them into predictors and outcomes. Multivariate analysis of variance (MANOVA) extends ANOVA to compare groups on multiple outcome variables simultaneously, testing whether the vector of means differs across groups while accounting for the correlations among the outcomes. MANOVA can be more powerful than separate ANOVAs, depending on how the outcomes correlate and how the group differences are patterned across them, and its single omnibus test controls the Type I error rate for the overall question; follow up tests on individual outcomes still need their own control of multiple testing.

Principal component analysis (PCA) reduces a large set of correlated variables into a smaller set of uncorrelated components that capture the maximum variance. It is used for data reduction, visualization, and as a preprocessing step before regression.

Factor analysis identifies underlying latent constructs (factors) that explain the correlations among a set of observed variables. Exploratory factor analysis (EFA) identifies the number and nature of factors, while confirmatory factor analysis (CFA) tests whether a hypothesized factor structure fits the data.

Structural equation modeling (SEM) combines factor analysis with path analysis to model complex networks of relationships among observed and latent variables, testing whether the proposed causal structure is consistent with the data.

Discriminant analysis classifies observations into groups based on multiple predictor variables, finding the combination of predictors that best separates the groups. It counts as a multivariate method because it is MANOVA turned around: the continuous variables that a MANOVA compares as outcomes become the variables that separate the groups, which is why it is a common follow up to a significant MANOVA.

Canonical correlation analysis examines the relationship between two sets of variables, finding the linear combinations within each set that are maximally correlated with each other.

Use multivariate methods when you have two or more dependent variables that are measured on the same subjects and are expected to be correlated. MANOVA is appropriate when comparing groups on multiple outcomes. PCA and factor analysis are appropriate for data reduction, scale development, and identifying latent constructs. SEM is appropriate for testing complex causal theories involving multiple constructs and pathways. Discriminant analysis is appropriate for classification problems with multiple predictors and a categorical outcome.

A common mistake is confusing multivariate analysis (multiple outcomes) with multivariable analysis (multiple predictors, one outcome). Using the term "multivariate" to describe a regression with multiple predictors and a single outcome is technically incorrect, though this misuse is widespread (Hidalgo and Goodman, 2013). Another error is applying MANOVA when outcomes are not correlated, in which case separate ANOVAs are simpler and often more powerful. Some researchers apply PCA or factor analysis without checking the adequacy of the correlation matrix (using Bartlett's test or the KMO measure), extracting factors from data that are not suitable. Finally, using SEM to "prove" a causal model is inappropriate; SEM tests whether the data are consistent with the proposed model, but alternative models may fit equally well.

A psychologist measures four aspects of job satisfaction (satisfaction with pay, supervision, coworkers, and the nature of work) in 300 employees across three departments. Rather than running four separate ANOVAs, the researcher uses MANOVA to test whether the vector of four satisfaction scores differs across departments. The multivariate test (Pillai's trace = 0.18, F = 7.4, p < 0.001) is significant, indicating that departments differ on the combined outcome vector. Follow up discriminant analysis reveals that satisfaction with supervision is the strongest discriminator between departments.