Overview of Reviews

An overview of reviews, sometimes called an umbrella review, is a type of evidence synthesis that systematically compiles and summarizes the findings of multiple systematic reviews on a related topic or set of interventions. Rather than synthesizing primary studies, it synthesizes the reviews themselves, providing a high level summary of the existing synthesized evidence.

As the volume of systematic reviews has grown, clinicians, policymakers, and researchers increasingly face the challenge of navigating dozens or even hundreds of systematic reviews on related topics. A single clinical question, such as the effectiveness of exercise for depression, may be addressed by numerous systematic reviews focusing on different types of exercise, populations, comparison conditions, or outcome measures. Overviews of reviews address this problem by providing a comprehensive, organized summary of all relevant systematic reviews in a single document. They are particularly valuable for informing clinical practice guidelines, health technology assessments, and policy decisions that require a broad view of the evidence landscape. They can also identify gaps, inconsistencies, and areas of convergence across existing reviews.

Conducting an overview of reviews follows many of the same methodological steps as a systematic review, but the unit of analysis is the systematic review rather than the primary study. The process begins with a focused research question and a protocol that specifies the eligibility criteria for including reviews (typically requiring that included reviews meet the definition of a systematic review, with a systematic search and predefined inclusion criteria at minimum).

A comprehensive search is conducted across bibliographic databases to identify all relevant systematic reviews. The identified reviews are screened for eligibility, and data are extracted on their characteristics (search dates, number of included studies, populations covered, interventions examined), methods (meta analytic versus narrative synthesis, risk of bias assessment), and findings (effect estimates, strength of evidence conclusions). The methodological quality of each included review is assessed using tools such as AMSTAR 2 (A MeaSurement Tool to Assess systematic Reviews), which evaluates the rigor and transparency of the systematic review process.

The results are typically presented in a structured format, often organized by intervention, outcome, or population, that allows readers to compare findings across reviews. When reviews address the same question but reach different conclusions, the overview examines the reasons for discrepancies, such as differences in search dates, inclusion criteria, or analytical methods. Some overviews of reviews include quantitative summary tables showing the direction, magnitude, and quality of evidence from each review, though formal meta analysis of the review level results is uncommon because of the challenges of double counting primary studies that appear in multiple reviews.

The overlap of primary studies across included reviews is a unique methodological challenge. Many systematic reviews on a given topic include the same landmark trials, and simply pooling or comparing their effect estimates without accounting for this overlap can give misleading impressions of the volume and independence of the evidence. Reporting the degree of overlap is recommended, for example by the PRIOR reporting guideline for overviews of reviews (2022), and measures such as the corrected covered area, introduced by Pieper and colleagues in 2014, quantify it.

Conduct an overview of reviews when multiple systematic reviews exist on a broad clinical or research question, when you need to summarize the state of knowledge for guideline development or policy decisions, or when you want to identify discrepancies and gaps across existing reviews. An overview is not appropriate when few or no systematic reviews exist on the topic; in that case, conducting a new systematic review of primary studies is the correct approach.

A common mistake is failing to assess the methodological quality of included reviews, treating all systematic reviews as equally rigorous. Using AMSTAR 2 or a similar appraisal tool is essential for distinguishing high quality from low quality reviews. Another error is ignoring the overlap of primary studies across included reviews, which can lead to overestimating the volume of independent evidence. Some overviews include nonsystematic narrative reviews alongside systematic reviews, diluting the quality of the synthesis. Finally, drawing conclusions about specific interventions based on the number of reviews supporting them (a "vote counting" approach) rather than on the quality and consistency of the evidence is a methodological weakness.

A team developing clinical practice guidelines for the management of chronic pain commissions an overview of reviews to summarize the evidence on pharmacological and nonpharmacological interventions. Their search identifies 45 systematic reviews published between 2015 and 2024 covering exercise therapy, cognitive behavioral therapy, acupuncture, opioids, nonsteroidal anti inflammatory drugs, and several other interventions. They assess each review with AMSTAR 2, chart which primary studies each review includes to measure their overlap, and present the findings in a matrix organized by intervention and outcome (pain intensity, function, quality of life, adverse events). The overview reveals that exercise therapy is supported by multiple high quality reviews with consistent findings, while the evidence for acupuncture is mixed and based on reviews of lower methodological quality.