Archival Research

Archival research is a method of inquiry that involves analyzing existing records, documents, or datasets that were originally created for purposes other than the current study. Sources may include medical records, government archives, organizational files, historical documents, census data, judicial records, and institutional databases.

Archival research enables researchers to study phenomena that would be impossible, impractical, or unethical to investigate through prospective data collection. Historical events, long term trends, rare conditions, and deceased populations can all be studied through their documentary traces. Because the data already exist, archival research is often faster and less expensive than primary data collection. It is widely used in epidemiology (through medical records and registries), history, political science, sociology, and organizational studies. Bibliometric analysts also work from existing records, such as publication databases, to study how scholarly fields evolve.

The process begins with identifying an archive or dataset relevant to the research question. Researchers must evaluate the quality, completeness, and provenance of the available records. Unlike data collected specifically for research, archival data were created for administrative, clinical, legal, or commercial purposes, and their structure and content reflect those original purposes rather than the researcher's analytical needs.

Data extraction involves systematically reviewing records and coding the information relevant to the research question. For quantitative analyses, researchers create structured databases from the raw records, defining variables, coding categories, and handling missing data. For qualitative analyses, researchers may conduct thematic analysis, content analysis, or discourse analysis of textual documents.

Key methodological considerations include the completeness of the archive (are records missing, and if so, why?), the consistency of recording practices over time (have diagnostic criteria, administrative categories, or documentation standards changed?), and the accuracy of the original records (were they verified, or do they reflect self report, estimation, or clerical error?). Researchers must also address issues of access and confidentiality, since many archives contain personally identifiable information that requires ethical approval and data use agreements.

Archival research may be retrospective (studying past events through existing records) or involve the secondary analysis of previously collected research data. In either case, the researcher must be transparent about the limitations of working with data collected for a different purpose and the potential for biases that arise from selective preservation, recording practices, and data availability.

Archival research is appropriate when the research question involves historical events, long term trends, or rare phenomena that cannot be studied prospectively; when primary data collection would be prohibitively expensive, time consuming, or ethically problematic; and when existing records provide sufficient detail to address the research question. It is less appropriate when the research question requires data that were not recorded in the available archives, when the accuracy of the original records is questionable and cannot be verified, or when the research requires experimental manipulation or prospective observation.

A common error is treating archival data as equivalent to data collected under controlled research conditions, failing to account for the biases inherent in records created for other purposes. Medical records, for instance, may undercount conditions that are underdiagnosed or stigmatized. Administrative datasets may reflect organizational priorities rather than clinical reality.

Another mistake is ignoring changes in coding systems, diagnostic criteria, or recording practices over time, which can produce apparent trends that are really artifacts of administrative changes rather than true changes in the phenomenon under study. For example, US hospitals switched from ICD-9-CM to ICD-10-CM diagnosis codes in October 2015, which shifted the recorded counts of many conditions. Researchers should also be cautious about survivor bias in archives: records may be preserved selectively, with certain types of documents (or records of certain types of people) being more likely to survive than others. Finally, failing to obtain appropriate ethical approval for archival research involving identifiable data is both an ethical and a legal risk.

An epidemiologist studies the long term cardiovascular outcomes of soldiers exposed to Agent Orange during the Vietnam War by linking military deployment records with Medicare claims data from 1999 to 2019. The analysis identifies 48,000 exposed veterans and matches them with unexposed veterans on age, rank, and branch of service. After adjusting for smoking status and body mass index recorded in VA health assessments, the study finds a 23% higher hazard of ischemic heart disease in the exposed group (HR = 1.23, 95% CI: 1.14 to 1.33).