Bias

Bias is a systematic error in the design, conduct, analysis, or reporting of a study that produces results that consistently deviate from the truth in a particular direction. Unlike random error, which varies unpredictably and can be reduced by increasing sample size, bias affects results in a consistent way that cannot be eliminated by collecting more data.

Bias is the central concern of research methodology because it threatens the validity of study findings and the conclusions drawn from them. Biased studies can lead to incorrect clinical guidelines, ineffective policies, and wasted research resources. Systematic reviewers and meta analysts must assess the risk of bias in each included study to determine how much confidence to place in the pooled results. The entire infrastructure of research design, from randomization and blinding to allocation concealment and intention to treat analysis, exists to minimize bias. Understanding the many forms of bias and the strategies for preventing them is essential knowledge for every researcher, reviewer, and consumer of research evidence.

Bias can enter a study at virtually every stage. Selection bias occurs when the process of selecting or assigning participants creates systematic differences between comparison groups. Examples include inadequate randomization, self selection into treatment groups, and differential loss to follow up. Performance bias arises when participants or researchers behave differently because they know which treatment is being received, affecting outcomes through placebo effects, co interventions, or altered reporting. Detection bias occurs when the assessment of outcomes is influenced by knowledge of treatment assignment, leading to differential measurement between groups. Reporting bias occurs when the selection of reported outcomes is influenced by the results, such as when researchers publish only statistically significant findings.

Each type of bias operates through a specific mechanism, and the appropriate preventive strategy depends on the type of bias being addressed. Randomization guards against selection bias at baseline. Allocation concealment guards against selection bias at enrollment. Blinding reduces performance and detection bias. Intention to treat analysis, with appropriate handling of missing data, reduces attrition bias. Pre registration helps prevent and detect reporting bias.

Risk of bias is assessed using structured tools such as the Cochrane Risk of Bias tool (RoB 2, for randomized trials), the Newcastle Ottawa Scale (for observational studies), and the ROBINS-I tool (for non randomized studies of interventions). These tools guide reviewers through a systematic evaluation of each study's methods, domain by domain, each with its own scale: RoB 2 judges each domain as low risk, some concerns, or high risk, while ROBINS-I uses low, moderate, serious, or critical.

Awareness of bias is relevant at every stage of the research process, from study design through data analysis to the interpretation and reporting of results. Researchers should identify the potential sources of bias in their study design and implement appropriate safeguards. Systematic reviewers should assess risk of bias for every included study using validated tools. Consumers of research evidence should evaluate the potential for bias when interpreting study findings and making decisions based on them.

A common error is assuming that only large, obvious flaws introduce bias. Subtle biases, such as differences in how outcomes are measured or how missing data are handled, can have substantial effects on results. Another mistake is conflating bias with imprecision; a study can be very precise (narrow confidence intervals) but biased (the entire interval is shifted away from the truth), and vice versa.

Researchers sometimes claim that a study is "unbiased" because it used randomization, ignoring other sources of bias such as inadequate blinding, differential attrition, or selective outcome reporting. Randomization, together with allocation concealment, addresses only one type of bias (selection bias at baseline) and does not protect against other threats. Finally, reviewers sometimes use numerical quality scores (summing across bias domains) rather than domain specific assessments, which obscures important information about where the bias risks lie and how they might affect the specific outcomes of interest.

A systematic review evaluating the effectiveness of acupuncture for chronic pain includes 24 randomized trials. Risk of bias assessment using the original Cochrane tool reveals that 8 trials have high risk of performance bias (patients and practitioners were not blinded), 6 have high risk of detection bias (outcome assessors knew treatment assignment), and 4 have high risk of attrition bias (more than 30% of participants were lost to follow up with differential rates between groups). A sensitivity analysis excluding the high risk studies reduces the pooled effect size from a standardized mean difference of 0.45 to 0.22, suggesting that bias in the lower quality studies inflated the apparent treatment effect.