Expertise versus Bias in Evaluation: Evidence from the NIH

It is shown thatevaluators are both better informed and more biased about the quality of projects in their own area, and policies designed to limit bias by seeking impartial evaluators may reduce thequality of funding decisions.

Abstract

Evaluators with expertise in a particular field may have an informational advantage in separating good projects from bad. At the same time, they may also have personal preferences that impact their objectivity. This paper examines these issues in the context of peer review at the US National Institutes of Health. I show that evaluators are both better informed and more biased about the quality of projects in their own area. On net, the benefits of expertise weakly dominate the costs of bias. As such, policies designed to limit bias by seeking impartial evaluators may reduce the quality of funding decisions. (JEL D82, H51, I10, I23, O38)