Kruskal Wallis Test

The Kruskal Wallis test is a nonparametric statistical test used to determine whether there are statistically significant differences in the distributions (assessed through mean ranks; a difference can be read as a difference in medians only when the groups' distributions have the same shape and spread) of a continuous or ordinal dependent variable across three or more independent groups. It serves as the nonparametric alternative to one way analysis of variance (ANOVA) when the assumption of normality is not met. It does not remove the need for similar spreads: unequal spreads across groups can themselves produce a significant result.

Many research datasets contain continuous or ordinal variables that are not normally distributed, particularly when sample sizes are small, when outcomes are measured on ordinal scales (such as Likert items), or when the data contain substantial outliers. Applying ANOVA to such data can produce misleading results because the test's validity depends on distributional assumptions. The Kruskal Wallis test provides a valid alternative that does not require normality and is resistant to the influence of outliers. It is widely used in clinical research, behavioral science, ecology, and any field where researchers need to compare three or more groups without relying on parametric assumptions.

The Kruskal Wallis test operates on the ranks of the data rather than the raw values. All observations from all groups are combined into a single sample and ranked from smallest to largest. Tied values receive the average of the ranks they would have occupied. The test statistic (H) is then calculated based on the sum of ranks within each group, adjusted for the overall mean rank. The formula is H = (12 divided by N(N+1)) times the sum across groups of (R_j squared divided by n_j) minus 3(N+1), where N is the total number of observations, R_j is the sum of ranks in group j, and n_j is the number of observations in group j.

Under the null hypothesis (all groups are drawn from the same distribution), the test statistic H approximately follows a chi squared distribution with k minus 1 degrees of freedom, where k is the number of groups. A significant result (p value below the chosen alpha) indicates that at least one group differs from the others, but it does not specify which groups differ or by how much.

To identify which specific groups differ, post hoc pairwise comparisons are conducted using tests such as the Dunn test, the Steel Dwass test, or the Conover test, with appropriate corrections for multiple comparisons (such as Bonferroni or Benjamini Hochberg adjustments). Effect size for the Kruskal Wallis test can be reported as eta squared based on H, calculated as (H minus k plus 1) divided by (N minus k), or as epsilon squared, calculated as H divided by ((N squared minus 1) divided by (N + 1)), which simplifies to H divided by (N minus 1); the rank biserial correlation is used for pairwise comparisons.

Use the Kruskal Wallis test when comparing three or more independent groups on a continuous or ordinal dependent variable and when one or more of the following conditions apply: the data are not normally distributed (as assessed by Shapiro Wilk test, Q Q plots, or visual inspection), sample sizes are small and the central limit theorem cannot be relied upon, the dependent variable is measured on an ordinal scale rather than a true interval or ratio scale, or the data contain outliers that would unduly influence the ANOVA F test. If you have only two groups, use the Mann Whitney U test instead, which is the two group equivalent.

A common error is using the Kruskal Wallis test without following up with post hoc comparisons after obtaining a significant result, leaving unanswered the question of which groups actually differ. Another mistake is interpreting the Kruskal Wallis test as comparing medians: it tests whether values in some groups tend to be larger than in others (whether the mean ranks differ), and it compares medians only when the groups' distributions have the same shape and spread. If the distributions have different shapes or spreads across groups, a significant result may reflect differences in variability or skew rather than in medians. Some researchers also fail to check whether the assumption of independent observations is met (the test does not handle repeated measures or clustered data). Finally, applying the test to very small groups (fewer than 5 observations per group) can reduce statistical power to unacceptable levels.

A sports science researcher compares perceived exertion (measured on a 6 to 20 Borg scale) across four training protocols in 80 recreational runners (20 per group). A Shapiro Wilk test indicates significant departures from normality in three of the four groups. The Kruskal Wallis test yields H(3) = 14.8, p = 0.002, indicating significant differences among the groups. Post hoc Dunn tests with Bonferroni correction reveal that Protocol D produces significantly lower perceived exertion than both Protocol A (adjusted p = 0.003) and Protocol B (adjusted p = 0.01), while no other pairwise comparisons are significant.