A quantitative study on the effectiveness of peer review for academic journals
ScientometricsPublished 18 December 2012
Zheng Ma, Yuntao Pan, Zhenglu Yu, Jingting Wang, Jia Jia, Yishan Wu
Citations18
SJR quartileQ1
SJR score1.15
SNIP1.82
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A new indicator for academic journals, Effectiveness of Peer Review (EPR), is defined for evaluating the effectiveness of peer reviewing, and the EPR indicator was able to reflect accurately the academic impact of those journals.
Abstract
Abstract Peer review is a classic method in the field of quality analysis, but the effectiveness of peer reviewing has never been researched using quantity analysis indicators. A new indicator for academic journals, Effectiveness of Peer Review (EPR), is defined for evaluating the effectiveness of peer reviewing. If the assumption is valid, EPR could be a simple indicator of such effectiveness. In a sample experiment, 28 academic journals were tested, and the EPR indicator was able to reflect accurately the academic impact of those journals.
Keywords
Computer ScienceDecision Sciences
ScientometricsSleeping Beauties in science
565 Citations2004Anthony F. J. van Raan
The first extensive measurement of the occurrence of Sleeping Beauties in the science literature is reported, derived from the measurements an ‘awakening’ probability function and identified the ‘most extreme Sleeping Beauty so far’.
JAMAEffect on the Quality of Peer Review of Blinding Reviewers and Asking Them to Sign Their Reports
336 Citations1998Fiona Godlee, Catharine R. Galé +1 more
Neither blinding reviewers to the authors and origin of the paper nor requiring them to sign their reports had any effect on rate of detection of errors, and such measures are unlikely to improve the quality of peer review reports.
Cochrane Database of Systematic ReviewsEditorial peer review for improving the quality of reports of biomedical studies
310 Citations2007Tom Jefferson, Melanie Rudin +2 more
Little empirical evidence is available to support the use of editorial peer review as a mechanism to ensure quality of biomedical research and a large, well-funded programme of research on the effects of editorialpeer review should be urgently launched.
JAMAMeasuring the Quality of Editorial Peer Review
226 Citations2002Tom Jefferson
Until the objectives of peer-review are properly defined, it will remain almost impossible to assess or improve its effectiveness, and research needed to understand the broader effects of peer review poses many methodologic problems and would require the cooperation of many parts of the scientific community.
Advances in Health Sciences EducationImpartial Judgment by the “Gatekeepers” of Science: Fallibility and Accountability in the Peer Review Process
204 Citations2003Mohammadreza Hojat, Joseph S. Gonnella +1 more
This article describes several pitfalls that can impede reviewers' impartial judgement and suggests that increasing reviewers' awareness of the pitfalls, accountability, and vigilance can improve fairness in the peer review process.
ScientometricsBetter late than never? On the chance to become highly cited only beyond the standard bibliometric time horizon
183 Citations2003Wolfgang Glänzel, Balázs Schlemmer +1 more
This study is focussed on a large-scale analysis of the citation history of all papers indexed in the 1980 annual volume of the Science Citation Index to analyse whether the share of delayed recognition papers is significant and whether such papers are typical of the work of their authors at that time.
Journal of the American Society for Information Science and TechnologyThe decline in the concentration of citations, 1900–2007
179 Citations2009Vincent Larivière, Yves Gingras +1 more
Changes in the concentration of citations received by papers published between 1900 and 2005 are analyzed and show that, contrary to what was reported by Evans, the dispersion of citations is actually increasing.
Research in Science EducationThe Paradox of Peer Review: Admitting too Much or Allowing too Little?
96 Citations2002Margaret Eisenhart
Environmental Science & TechnologyThe fate of toxic airborne pollutants
77 Citations1988W. H. Schroeder, Douglas A. Lane
Mediante modelos matematicos especificos se determinan las cargas contaminantes, que incluyen volumenes de produccion o emision del contaminante.
The Open Ecology JournalBang for Your Buck: Rejection Rates and Impact Factors in Ecological Journals
75 Citations2008Lonnie W. Aarssen, Tom Tregenza +4 more
Science and Engineering EthicsEvidence for the effectiveness of peer review
63 Citations1997Robert H. Fletcher, Suzanne W. Fletcher
In a study of whether manuscripts are improved by peer review and editing, articles published in Annals of Internal Medicine were improved in 33 of 34 dimensions of reporting quality, but published articles still had room for improvement.
ScientometricsScientometric indicators: peer-review, bibliometric methods and conflict of interests
48 Citations2010Primož Južnič, Stojan Pečlin +4 more
It was shown that while scientometric indicators can hardly replace the peer- review system as the ultimate decision-making and support system, they can reveal its weaknesses on one hand and on the other can verify peer-review scores and minimize conflict of interest if necessary.
Bulletin of the American Meteorological SocietyRejection Rates for Journals Publishing in the Atmospheric Sciences
36 Citations2010David M. Schultz
Science and Engineering EthicsReferees, editors, and publication practices: Improving the reliability and usefulness of the peer review system
23 Citations1997Domenic V. Cicchetti
Hydrological Sciences JournalEditorial—The peer-review system: prospects and challenges
20 Citations2005Zbigniew W. Kundzewicz, Demetris Koutsoyiannis
Delayed Recognition:Sleeping Beauties in Science——A Case Study of a Superstring Theory Paper
4 Citations2009Liming Liang
American Journal of RoentgenologyNew category of peer-review decision: rejection with opportunity to revise and resubmit.
4 Citations1995R N Berk, David P. Friedman
Environmental Science & TechnologyEditorial: Peer review: A foundation of science.
2 Citations1988William H. Glaze
Jisuanji yingyong yanjiuBiblio-metrilogical analysis on development trend of computer science in China
2 Citations2007Haiyan Yu
The paper analyzed the status quo and development trend of computer science in China based on bibliometrics to study the trend of papers in computer science, the development of main subfields and the influence of the periodicals that publish these papers.