Discriminant Validity

Discriminant validity is a form of construct validity that provides evidence that measures of theoretically distinct constructs are empirically distinguishable from one another. It is demonstrated when a measure does not correlate too highly with measures of different constructs, confirming that the instrument captures a unique dimension rather than simply duplicating what another instrument measures. Discriminant validity is the complement of convergent validity, which shows that measures of related constructs correlate as theory predicts.

In behavioral, social, and health sciences, researchers frequently measure multiple constructs using self report instruments that may share common method variance or overlapping item content. If a measure of anxiety correlates as highly with a measure of depression as it does with another measure of anxiety, the two instruments may not be measuring distinct constructs, and research that claims to distinguish between them is on shaky ground. Discriminant validity evidence is essential for establishing that a new instrument adds something unique to the measurement landscape. Without it, researchers risk constructing theories about relationships between constructs that are actually relationships between differently labeled versions of the same construct.

Discriminant validity is typically assessed using correlation analysis or confirmatory factor analysis (CFA). The simplest approach examines whether the correlation between two measures that should be unrelated (or only weakly related) is significantly lower than the correlation between measures of the same construct. For example, a new measure of work engagement should correlate modestly with a measure of job satisfaction (a related but distinct construct) and weakly with a measure of physical flexibility (an unrelated construct).

The Campbell and Fiske multitrait multimethod (MTMM) matrix provides a more rigorous framework. Researchers measure multiple traits using multiple methods and examine the resulting correlation matrix. Discriminant validity is supported when correlations between different traits measured by the same method (heterotrait monomethod) are lower than correlations between the same trait measured by different methods (monotrait heteromethod). This approach also disentangles trait variance from method variance.

In confirmatory factor analysis, discriminant validity is assessed by comparing a model in which two constructs are allowed to correlate freely with a model in which their correlation is fixed at 1.0 (implying they are the same construct). A significantly better fit for the freely estimated model supports discriminant validity. The Fornell Larcker criterion provides a specific benchmark: the average variance extracted (AVE) for each construct should exceed the squared correlation between that construct and every other construct. The heterotrait monotrait (HTMT) ratio is a more recently developed criterion, proposed by Henseler, Ringle and Sarstedt (2015) after simulations in variance based structural equation modeling showed that the Fornell Larcker criterion often fails to detect a lack of discriminant validity. HTMT values below 0.85, or below 0.90 for conceptually very similar constructs, are commonly taken as evidence of discriminant validity.

Discriminant validity should be assessed whenever a researcher develops or validates a new measurement instrument, particularly when the construct is conceptually close to existing constructs. It is also important in structural equation modeling, where the interpretation of paths between latent variables depends on the distinctiveness of those variables. Discriminant validity assessment is standard practice in instrument development studies and in measurement validation studies. It is not a standalone assessment but should be evaluated alongside convergent validity, content validity, and criterion validity as part of a comprehensive validation strategy.

A common error is declaring discriminant validity based solely on the observation that two constructs' correlation is "not too high" without applying a specific criterion or statistical test. Another mistake is testing discriminant validity only against measures of clearly unrelated constructs (such as validating a depression scale against a math ability test), where low correlation is expected and uninformative. The meaningful test is against measures of related but theoretically distinct constructs. Researchers sometimes confuse discriminant validity with specificity (a diagnostic accuracy term) or with discriminant function analysis (a classification method), which are entirely different concepts. Reporting only the correlation coefficient without considering measurement error can lead to underestimation of the true correlation between constructs, making discriminant validity appear stronger than it actually is.

Researchers validating a new 10 item measure of academic burnout assess discriminant validity by examining its correlations with established measures. The new scale correlates 0.72 with another burnout measure (convergent validity), 0.45 with a depression scale (moderate correlation with a related but distinct construct), and 0.12 with a measure of extraversion (low correlation with an unrelated construct). CFA comparing a two factor model (burnout and depression as separate factors) to a one factor model (combined) shows significantly better fit for the two factor model (delta chi square = 42.3, df = 1, p < 0.001). In the two factor model, the correlation between the burnout and depression factors is 0.53, higher than the observed 0.45 because the factor correlation is corrected for measurement error. Its square (0.28) is lower than the AVE for both burnout (0.54) and depression (0.61), satisfying the Fornell Larcker criterion and supporting discriminant validity.