Data Extraction

Data extraction in the context of evidence synthesis is the systematic process of collecting predefined information from each study included in a systematic review, using a standardized form or template. Extracted data typically encompass study design characteristics, participant demographics, intervention and comparator details, outcome definitions, measurement time points, and quantitative results. It is one of the most labor intensive and methodologically critical steps in the systematic review process.

Data extraction is the bridge between study selection and evidence synthesis. Every number that enters a meta analysis, every characteristic that populates an evidence table, and every comparison that informs a clinical guideline originates from data extraction. Errors at this stage propagate directly into pooled effect estimates, potentially altering conclusions about treatment efficacy, diagnostic accuracy, or intervention safety. Systematic review guidance emphasizes standardized forms and a second person checking every extraction. Cochrane's conduct standards (MECIR) make independent extraction of outcome data by at least two people mandatory and call it highly desirable for study characteristics, and the Joanna Briggs Institute also calls for two reviewers extracting independently; Cochrane's guidance for rapid reviews accepts one reviewer extracting with a second checking the data for correctness and completeness. The accuracy and completeness of extracted data determine whether the review's conclusions are trustworthy.

The process begins before extraction itself, with the development and pilot testing of an extraction form. The form is designed to capture all data elements specified in the review protocol, organized in a logical sequence. Common elements include bibliographic information, study design, setting, sample size, eligibility criteria, baseline participant characteristics (age, sex, disease severity), intervention details (dose, duration, mode of delivery), comparator details, primary and secondary outcomes (definitions, measurement instruments, time points), and effect estimates with their precision (means and standard deviations, odds ratios and confidence intervals, hazard ratios, or event counts).

The form is pilot tested on a representative sample of included studies by all reviewers who will participate in extraction. This pilot identifies ambiguities in the form, inconsistencies in how reviewers interpret study reports, and data elements that need clarification or restructuring. Revisions are made before full extraction begins.

At least two reviewers independently extract data from each study, a practice known as dual extraction. Independent extraction reduces the risk of errors from misreading tables, transcribing numbers incorrectly, or making subjective judgments about ambiguous reporting. After independent extraction, the two sets of results are compared systematically. Discrepancies are resolved through discussion between the reviewers, reference to the original publication, or adjudication by a third reviewer. The rate and nature of discrepancies should be documented, as high disagreement rates may indicate problems with the extraction form or insufficient training.

When studies report incomplete data, reviewers may need to contact original authors, extract data from figures using digitization software, or calculate missing statistics from available information (for example, deriving standard deviations from confidence intervals or p values). These decisions should be documented in the review's methods section.

Paperguide's Data Extraction tool lets researchers define custom columns and pull data from multiple papers into one extraction table; extracted values should still be checked against the source reports.

Data extraction is a required step in every systematic review and meta analysis, regardless of the topic or discipline. It is also used in scoping reviews, rapid reviews, and evidence maps, though the depth and detail of extraction may vary by review type. Data extraction forms should be customized for each review based on the research question and the PICO (Population, Intervention, Comparator, Outcome) framework. For reviews that will include a meta analysis, extraction must capture sufficient quantitative detail to calculate or verify effect sizes.

A common error is beginning extraction without pilot testing the form, leading to inconsistent data collection that becomes apparent only after dozens of studies have been extracted. Another frequent mistake is relying on a single reviewer for extraction, which is faster but misses errors at a rate that can materially affect meta analytic results. Researchers sometimes extract only the data reported in abstracts without examining the full text, missing important details, corrections, or supplementary data. Failing to extract data on potential effect modifiers (such as patient age, disease severity, or follow up duration) limits the ability to conduct subgroup analyses or meta regression. Finally, inconsistent handling of studies that report outcomes at multiple time points or in multiple formats can introduce confusion and errors into the analysis dataset.

Two reviewers independently extract hazard ratios, 95% confidence intervals, median follow up duration, and patient demographics from 38 randomized trials of immunotherapy in non small cell lung cancer. They use a piloted electronic form with dropdown menus for categorical variables and numeric fields with range checks for quantitative data. Comparison of extractions reveals 14 discrepancies across 38 studies, most involving interpretation of subgroup data versus overall results. After resolution through discussion and review of the original papers, the extracted data are entered into Review Manager for meta analysis.