Peer Review Quality and Transparency of the Peer-Review Process in Open Access and Subscription Journals
PLoS ONEPublished 29 January 2016Open access
Jelte M. Wicherts
Citations158
SJR quartileQ1
SJR score0.80
SNIP1.07
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The tool to assess transparency of the peer-review process at academic journals shows promising reliability and validity, and can be seen as an indicator of peer- review quality allowing the tool to be used to predict academic quality in new journals.
Abstract
The tool to assess transparency of the peer-review process at academic journals shows promising reliability and validity. The transparency of the peer-review process can be seen as an indicator of peer-review quality allowing the tool to be used to predict academic quality in new journals.
Keywords
Decision Sciences
ScienceWho's Afraid of Peer Review?
930 Citations2013John Bohannon
Dozens of open- access journals targeted in an elaborate Science sting accepted a spoof research article, raising questions about peer-review practices in much of the open-access world.
ScientometricsComparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups
609 Citations2006Anthony F. J. van Raan
Characteristics of the statistical correlation between the Hirsch (h-) index and several standard bibliometric indicators are presented, as well as with the results of peer review judgment, which show that the h-index and the bibliometry ‘crown indicator’ both relate in a quite comparable way with peer judgments.
Journal of the American Society for Information Science and TechnologyPeer review in a changing world: An international study measuring the attitudes of researchers
399 Citations2012Adrian Mulligan, Louise Hall +1 more
The study found that the peer review process is highly regarded by the vast majority of researchers and considered by most to be essential to the communication of scholarly research.
ScientometricsAdvanced bibliometric methods as quantitative core of peer review based evaluation and foresight exercises
383 Citations1996Anthony F. J. van Raan
An overview of the potentials and limitations of bibliometric methods for the assessment of strengths and weaknesses in research performance, and for monitoring scientific developments, and how both approaches can be combined to a broader and powerful methodology to observe scientific advancement and the role of actors.
BMC MedicineAnatomy of open access publishing: a study of longitudinal development and internal structure
366 Citations2012Mikael Laakso, Bo‐Christer Björk
OA journal publishing is disrupting the dominant subscription-based model of scientific publishing, having rapidly grown in relative annual share of published journal articles during the last decade.
JAMAEffect on the Quality of Peer Review of Blinding Reviewers and Asking Them to Sign Their Reports
336 Citations1998Fiona Godlee, Catharine R. Galé +1 more
Neither blinding reviewers to the authors and origin of the paper nor requiring them to sign their reports had any effect on rate of detection of errors, and such measures are unlikely to improve the quality of peer review reports.
JAMAWhat Makes a Good Reviewer and a Good Review for a General Medical Journal?
266 Citations1998Nick Black
The characteristics of reviewers studied did not identify those who performed high-quality reviews, and reviewers might be advised that spending longer than 3 hours on a review on average did not appear to increase review quality as rated by editors and authors.
JAMAMeasuring the Quality of Editorial Peer Review
226 Citations2002Tom Jefferson
Until the objectives of peer-review are properly defined, it will remain almost impossible to assess or improve its effectiveness, and research needed to understand the broader effects of peer review poses many methodologic problems and would require the cooperation of many parts of the scientific community.
PLoS ONEA Reliability-Generalization Study of Journal Peer Reviews: A Multilevel Meta-Analysis of Inter-Rater Reliability and Its Determinants
221 Citations2010Lutz Bornmann, Rüdiger Mutz +1 more
According to the first meta-analysis for the inter-rater reliability (IRR) of journal peer reviews, the IRR of peer assessments is quite limited and needs improvement (e.g., reader system).
Research PolicyComparative analysis of a set of bibliometric indicators and central peer review criteria
189 Citations1998Ed J. Rinia, Thed N. van Leeuwen +2 more
JAMAAuthor Perception of Peer Review
141 Citations2002Ellen J Weber, Patricia Katz +2 more
Overall satisfaction was highly associated with acceptance of the manuscript for publication but not with quality rating of reviews, and contributor satisfaction with peer review was modest.
NaturePublishers withdraw more than 120 gibberish papers
123 Citations2014Richard Van Noorden
Conference proceedings removed from subscription databases after scientist reveals that they were computer-generated.
JAMAReliability of Editors' Subjective Quality Ratings of Peer Reviews of Manuscripts
120 Citations1998Michael L. Callaham
Subjective editor ratings of individual reviewers were moderately reliable and correlated with reviewer ability to report manuscript flaws, but these variables were poorly correlated with editors' ratings of review quality or the reviewer's ability to detect flaws in a fictitious manuscript.
Journal of InformetricsA longitudinal comparison of citation rates and growth among open access journals
117 Citations2013David Solomon, Mikael Laakso +1 more
Citation averages of OA journals funded by other means continue to lag well behind OA Journals funded by APCs and subscription journals, and it is hypothesized this is less an issue of quality than due to the fact that such journals are commonly published in languages other than English and tend to be located outside the four major publishing countries.
BMC Medical Research MethodologyViews on the peer review system of biomedical journals: an online survey of academics from high-ranking universities
102 Citations2013Roger Ho, Kwok‐Kei Mak +4 more
Female academics, experience of personal attacks and imposition of unnecessary references by reviewers were related to disagreement about fairness of the peer review system of biomedical journals.
Frontiers in Computational NeuroscienceLetting the daylight in: Reviewing the reviewers and other ways to maximize transparency in science
73 Citations2012Jelte M. Wicherts, Rogier Kievit +2 more
This work argues for the implementation of peer-reviewed peer review, transparent editorial hierarchies, and online data publication, arguing that the proposed system leads to better reviews, meritocrats, and a higher degree of replicability of statistical analyses.
PLoS ONEThe Validation of Peer Review through Research Impact Measures and the Implications for Funding Strategies
49 Citations2014Stephen A. Gallo, Afton S. Carpenter +6 more
A retrospective analysis of peer review data of 2,063 applications submitted to a particular research program and the bibliometric output of the resultant 227 funded projects over an 8-year period found peer review scores associated with applications were found to be moderately correlated with the total time-adjusted citation output of funded projects, although a high degree of variability existed in the data.
ScientometricsScientometric indicators: peer-review, bibliometric methods and conflict of interests
48 Citations2010Primož Južnič, Stojan Pečlin +4 more
It was shown that while scientometric indicators can hardly replace the peer- review system as the ultimate decision-making and support system, they can reveal its weaknesses on one hand and on the other can verify peer-review scores and minimize conflict of interest if necessary.
ScienceAs Open Access Explodes, How to Tell the Good From the Bad and the Ugly?
16 Citations2012Martin Enserink
At a meeting last month, representatives from science funding agencies, publishers, librarians, and journal editors discussed two proposed yardsticks for quality, but failed to come up with clear answers.
Communications of the Association for Information SystemsOpen Access: The Whipping Boy for Problems in Scholarly Publishing
13 Citations2015Danny Kingsley, Mary Anne Kennan
The Charleston AdvisorFive Scholarly Open Access Publishers
11 Citations2012Jeffrey Beall
This review critically examines five international scholarly publishers that publish academic journals using the gold (author pays) Open Access model, and covers four predatory publishers, Academy Publish, BioInfo, ScienceDomain International, and Scientific Research Publishing, and one legitimate publisher, AOSIS Open Journals.