Network Meta Analysis

Network meta analysis (NMA), also known as mixed treatment comparison, is a statistical technique that simultaneously compares three or more interventions by combining both direct evidence (from head to head trials) and indirect evidence (inferred through a common comparator) within a connected network of studies. It produces relative effect estimates for every possible pairwise comparison among all included interventions and can rank them by their probability of being the best option.

Clinical decisions often require choosing among multiple treatment options, but direct head to head comparisons exist for only a fraction of possible pairings. Network meta analysis fills this gap by combining all available trial evidence into a single coherent analysis, providing estimates for comparisons that have never been directly studied. It is now a widely used methodology in HTA submissions, clinical guideline development, and formulary decisions. NICE, Canada's Drug Agency (formerly CADTH), and other HTA bodies accept NMAs when head to head evidence is lacking, and the PRISMA extension for network meta analyses (PRISMA-NMA, 2015) provides reporting standards for these analyses.

The foundation of NMA is the transitivity assumption: the trials making different comparisons are similar in the characteristics that modify treatment effects (effect modifiers such as population, disease severity, dosing, or outcome definitions). Only then can A vs C trials and B vs C trials, with C serving as the bridge, give a valid indirect estimate of A versus B. Its statistical counterpart is the consistency assumption: the relative effect of treatment A versus treatment B should be the same whether it is estimated from direct comparisons (A vs B trials) or from indirect evidence via a common comparator. This assumption allows the combination of direct and indirect evidence into a single pooled estimate for each comparison.

The analysis begins with constructing a network diagram, where each node represents a treatment and each edge represents a set of trials comparing two treatments. The thickness of each edge often reflects the number of trials or participants. The network must be connected, meaning that every treatment can be linked to every other treatment through a chain of direct comparisons.

Statistical models for NMA can be frequentist (using multivariate meta analysis) or Bayesian (using Markov chain Monte Carlo methods). Bayesian approaches are more common and provide several advantages, including natural incorporation of prior information, direct estimation of ranking probabilities, and flexible handling of complex network structures. The model estimates the relative effects of all treatments compared to a common reference and uses these to derive all pairwise comparisons.

Results are presented as relative effect estimates (with credible or confidence intervals) for every pair of treatments, often displayed in a league table. Ranking can be summarized using the surface under the cumulative ranking curve (SUCRA) or the probability of being the best treatment, though rankings should be interpreted cautiously because small differences in effect can lead to large changes in rank.

The consistency assumption is tested by comparing direct and indirect evidence wherever closed loops exist in the network. The node splitting method compares the direct and indirect estimates for each comparison, and significant inconsistency flags potential problems such as differences in study populations, intervention definitions, or bias. Because these tests have low power, finding no significant inconsistency does not prove that the network is consistent, so transitivity still has to be judged by comparing the trials' characteristics. Sensitivity analyses and subgroup analyses explore the influence of study characteristics, risk of bias, and network structure on the results.

Use NMA when three or more interventions are relevant to a clinical question, when a connected network of randomized trials exists, and when decision makers need comparative information for all available options. It is widely used in HTA submissions, clinical guideline development, and systematic reviews that address a clinical question with multiple treatment alternatives. NMA should not be used when the network is disconnected (no pathway links all treatments), when the transitivity assumption is implausible (for example, when trials in different parts of the network differ systematically in effect modifiers such as population or design), or when the evidence base is too thin (very few trials with small sample sizes).

A common error is ignoring the consistency assumption or failing to test it, which can produce misleading estimates when direct and indirect evidence disagree. Another mistake is placing excessive emphasis on treatment rankings without considering the uncertainty around them; two treatments with similar effects can swap positions with slight changes in the data. Some researchers use NMA on networks where the transitivity assumption (studies comparing different treatments are similar in the characteristics that modify treatment effects) is violated due to major differences in populations, dosing, or outcomes across different parts of the network. Analyzing multi arm trials as if their comparisons were independent, without accounting for the correlated treatment effects within a trial, produces incorrect estimates, whether the model is frequentist or Bayesian. Finally, presenting NMA results without a thorough assessment of the certainty of evidence (using frameworks such as CINeMA or GRADE NMA) leaves decision makers without the information they need to weigh the findings appropriately.

In a hypothetical example, a systematic review identifies 48 RCTs comparing eight treatments for chronic insomnia: five pharmacological agents, two behavioral therapies (CBT for insomnia and brief behavioral therapy), and placebo. A Bayesian NMA using a random effects model produces estimates for all 28 pairwise comparisons. CBT for insomnia is ranked first for sleep onset latency reduction (SUCRA = 92%), followed by drug D (SUCRA = 78%) and drug B (SUCRA = 71%). Consistency testing reveals no significant inconsistency in any of the closed loops, and the review team judges that trials across the network are similar in their main effect modifiers. Because rankings can shift with small differences in effect, the guideline panel reads them alongside the league table's effect estimates, their credible intervals, and the certainty of the evidence, and recommends CBT for insomnia as first line treatment.