An Examination of Sources of Peer-Review Bias
Generate an AI Snapshot to get a quick, structured summary of this paper.
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
Peer-review ratings of 1,983 posters submitted for three annual conferences of a professional society were examined for evidence of bias and differences in reliability and acceptance decisions were examined.
Abstract
Peer-review ratings of 1,983 posters submitted for three annual conferences of a professional society were examined for evidence of bias. Hypotheses derived from the literature on the better-than-average effect were tested by analyzing 7,383 sets of ratings. Reviewers who authored posters gave lower average ratings than reviewers who did not author posters. Posters having authorship that included at least one reviewer received higher ratings than those having only nonreviewing authors. Reviewers' experience and professional role were also explored as biasing factors. The ratings were converted into z scores, and differences in reliability and acceptance decisions were examined. Implications for current peer-review practices are discussed.