The Emergence of Implicit Criteria Actually Used by Reviewers of Qualitative Research Articles

Abstract

This article brings to light the actual evaluation practices of reviewers when assessing qualitative manuscripts. The authors conducted the first empirical research entirely based on reviewer reports for a journal on management sciences over a 28-year period. Content analysis of 474 reviewer reports written by 56 reviewers identified 19 critical points and 10 criteria, making up a synthetic table of 190 possible cells, 51 of which proved to be actually used by reviewers. These findings are then compared with the quality criteria identified in the extant U.S. literature. Comparison reveals some shared quality criteria but also throws new light on a number of discrepancies. Analysis over time shows marked evolution from emphasis on internal validity criteria over the first 10 years toward emphasis on external validity criteria in the past 10 years. Factors ensuring reliability and replicability of the results of this research are discussed at length.