How to Make More Published Research True
PLoS MedicinePublished 21 October 2014Open access
John P. A. Ioannidis
Citations782
SJR quartileQ1
SJR score4.28
SNIP3.21
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
Abstract
In a 2005 paper that has been accessed more than a million times, John Ioannidis explained why most published research findings were false. Here he revisits the topic, this time to address how to improve matters. Please see later in the article for the Editors' Summary.
Keywords
Decision Sciences
Review of General PsychologyConfirmation Bias: A Ubiquitous Phenomenon in Many Guises
7,107 Citations1998Raymond S. Nickerson
Psychological ScienceFalse-Positive Psychology
6,712 Citations2011Joseph P. Simmons, Leif D. Nelson +1 more
It is shown that despite empirical psychologists’ nominal endorsement of a low rate of false-positive findings, flexibility in data collection, analysis, and reporting dramatically increases actual false- positive rates, and a simple, low-cost, and straightforwardly effective disclosure-based solution is suggested.
CHANCEWhy Most Published Research Findings Are False
4,827 Citations2005John P. A. Ioannidis
Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true.
National Academies Press eBooksClinical Practice Guidelines We Can Trust
3,070 Citations2011Board on Health Care Services
Clinical Practice Guidelines The authors Can Trust shows how clinical practice guidelines can enhance clinician and patient decision-making by translating complex scientific research findings into recommendations for clinical practice that are relevant to the individual patient encounter, instead of implementing a one size fits all approach to patient care.
NatureRaise standards for preclinical cancer research
2,930 Citations2012C. Glenn Begley, Lee M. Ellis
C. Glenn Begley and Lee M. Ellis propose how methods, publications and incentives must change if patients are to benefit.
Nature Reviews Drug DiscoveryBelieve it or not: how much can we rely on published data on potential drug targets?
1,924 Citations2011Florian Prinz, Thomas Schlange +1 more
It is indicated that the limitations of the predictivity of disease models and also that the validity of the targets being investigated is frequently questionable, which is a crucial issue to address if success rates in clinical trials are to be improved.
BMJEvidence based medicine: a movement in crisis?
1,735 Citations2014Trisha Greenhalgh, Jeremy Howick +2 more
A preliminary agenda for the evidence based medicine movement’s renaissance is offered, refocusing on providing useable evidence that can be combined with context and professional expertise so that individual patients get optimal treatment.
NatureScientific method: Statistical errors
1,544 Citations2014Regina Nuzzo
It turned out that the problem was not in the data or in Motyl's analyses, it lay in the surprisingly slippery nature of the P value, which is neither as reliable nor as objective as most scientists assume.
EpidemiologyWhy Most Discovered True Associations Are Inflated
1,540 Citations2008John P. A. Ioannidis
Potential approaches to this problem include being cautious about newly discovered effect sizes, using analytical methods that correct for the anticipated inflation, ignoring the magnitude of the effect (if not necessary), conducting large studies in the discovery phase, using strict protocols for analyses, pursuing complete and transparent reporting of all results, placing emphasis on replication, and being fair with interpretation of results.
ScienceReproducible Research in Computational Science
1,442 Citations2011Roger D. Peng
This work states that reproducibility has the potential to serve as a minimum standard for judging scientific claims when full independent replication of a study is not possible.
The LancetHow to increase value and reduce waste when research priorities are set
1,434 Citations2014Iain Chalmers, Michael B. Bracken +7 more
This report discusses how avoidable waste can be considered when research priorities are set and recommends ways to improve the yield from basic research, and the transparency of processes by which funders prioritise important uncertainties should be increased.
The LancetReducing waste from incomplete or unusable reports of biomedical research
1,359 Citations2014Paul Glasziou, Douglas G. Altman +7 more
The high amount of waste at all stages in medical research warrants future investment in the monitoring of and research into reporting of research, and active implementation of the findings to ensure that research reports better address the needs of the range of research users.
NatureNepotism and sexism in peer-review
1,357 Citations1997Christine Wennerås, Agnes E. Wold
In the first-ever analysis of peer-review scores for postdoctoral fellowship applications, the system is revealed as being riddled with prejudice.
NatureReplicating genotype–phenotype associations
1,350 Citations2007Stephen J. Chanock, Teri A. Manolio +35 more
What constitutes replication of a genotype–phenotype association, and how best can it be achieved, is investigated.
Perspectives on Psychological ScienceScientific Utopia
1,336 Citations2012Brian A. Nosek, Jeffrey R. Spies +1 more
Strategies for improving scientific practices and knowledge accumulation are developed that account for ordinary human motivations and biases and can reduce the persistence of false findings.
NaturePolicy: NIH plans to enhance reproducibility
1,285 Citations2014Francis S. Collins, Lawrence A. Tabak
Francis S. Collins and Lawrence A. Tabak discuss initiatives that the US National Institutes of Health is exploring to restore the self-correcting nature of preclinical research.
NatureA call for transparent reporting to optimize the predictive value of preclinical research
1,213 Citations2012Story C. Landis, Susan Amara +34 more
The main workshop recommendation is that at a minimum studies should report on sample-size estimation, whether and how animals were randomized, whether investigators were blind to the treatment, and the handling of data.
PLoS ONESystematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias — An Updated Review
1,087 Citations2013Kerry Dwan, Carrol Gamble +3 more
Direct empirical evidence for the existence of study publication bias and outcome reporting bias is shown and there is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported.
European Journal of Clinical InvestigationA catalogue of reporting guidelines for health research
1,048 Citations2009Iveta Simera, David Moher +3 more
ScienceWho's Afraid of Peer Review?
930 Citations2013John Bohannon
Dozens of open- access journals targeted in an elaborate Science sting accepted a spoof research article, raising questions about peer-review practices in much of the open-access world.
The LancetIncreasing value and reducing waste: addressing inaccessible research
907 Citations2014An‐Wen Chan, Fujian Song +7 more
Three main actions are warranted: academic institutions and funders should reward investigators who fully disseminate their research protocols, reports, and participant-level datasets, and standards for the content of protocols and full study reports should be rigorously developed and adopted for all types of health research.
Proceedings of the National Academy of SciencesRevised standards for statistical evidence
819 Citations2013Valen E. Johnson
Modifications of common standards of evidence are proposed to reduce the rate of nonreproducibility of scientific research by a factor of 5 or greater and to correct the problem of unjustifiably high levels of significance.
e-publications@bond (Bond University)Biomedical research: Increasing value, reducing waste
797 Citations2014Macleod, Malcolm, Michie, Susan +7 more
New England Journal of MedicineThe ClinicalTrials.gov Results Database — Update and Key Issues
789 Citations2011Deborah A. Zarin, Tony Tse +3 more
The structure and contents of the results database are summarized, an update of relevant policies are provided, and how the data can be used to gain insight into the state of clinical research are shown.
PLoS ONE“Positive” Results Increase Down the Hierarchy of the Sciences
750 Citations2010Daniele Fanelli
These results support the scientific status of the social sciences against claims that they are completely subjective, by showing that, when they adopt a scientific approach to discovery, they differ from the natural sciences only by a matter of degree.
Perspectives on Psychological ScienceIs the Replicability Crisis Overblown? Three Arguments Examined
695 Citations2012Harold Pashler, Christine R. Harris
It is argued that there are no plausible concrete scenarios to back up such forecasts and that what is needed is not patience, but rather systematic reforms in scientific practice.
The LancetIncreasing value and reducing waste in biomedical research regulation and management
562 Citations2014Rustam Al‐Shahi Salman, Elaine Beller +7 more
Researchers and research managers should increase the efficiency of recruitment, retention, data monitoring, and data sharing in research through use of research designs known to reduce inefficiency, and further research should be done to learn how efficiency can be increased.
Quarterly Journal of Experimental PsychologyConfirmation Bias in a Simulated Research Environment: An Experimental Study of Scientific Inference
522 Citations1977Clifford R. Mynatt, Michael E. Doherty +1 more
PLoS MedicineWhy Current Publication Practices May Distort Science
512 Citations2008Neal S. Young, John P. A. Ioannidis +1 more
John Ioannidis and colleagues argue that the current system of publication in biomedical research provides a distorted view of the reality of scientific data.
BMJCochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review
442 Citations2006Anders W Jørgensen, Jørgen Hilden +1 more
Industry supported reviews of drugs should be read with caution as they were less transparent, had few reservations about methodological limitations of the included trials, and had more favourable conclusions than the corresponding Cochrane reviews.
Psychological InquiryScientific Utopia: I. Opening Scientific Communication
391 Citations2012Brian A. Nosek, Yoav Bar‐Anan
This work outlines a path that moves away from the existing model of scientific communication to improve the efficiency in meeting the purpose of public science—knowledge accumulation and calls for six changes.
EpidemiologyThe False-positive to False-negative Ratio in Epidemiologic Studies
330 Citations2011John P. A. Ioannidis, Robert E. Tarone +1 more
The ratio of false-positive to false-negative findings (FP:FN ratio) is an informative metric that warrants further evaluation and has implications for what is the acceptable FP:FN ratios and for how the results of published epidemiologic studies should be presented and interpreted.
American Journal of EpidemiologyReproducible Epidemiologic Research
300 Citations2006Roger D. Peng, Francesca Dominici +1 more
A standard for reproducibility is outlined and methods for reproducible research are proposed and implemented by use of a case study in air pollution and health.
PLoS ONEToward Reproducible Computational Research: An Empirical Analysis of Data and Code Policy Adoption by Journals
264 Citations2013Victoria Stodden, Peixuan Guo +1 more
A predictive model of open data and code policy adoption as a function of impact factor and publisher is built and it is found higher impact journals more likely to have Open Data and Code Policy Day and scientific societies more likely than commercial publishers to have openData and code policies.
BMJOverlapping meta-analyses on the same topic: survey of published studies
241 Citations2013Konstantinos C. Siontis, Tina Hernandez‐Boussard +1 more
There is a waste of efforts with many topics covered by multiple overlapping meta-analyses of randomized trials published on the same topic, according to the overall picture.
New England Journal of MedicineThe Journal's Policy on Cost-Effectiveness Analyses
237 Citations1994Jerome P. Kassirer, Marcia Angell
Concern about cost now dominates many decisions about the use of drugs and other therapeutic interventions, and published economic analyses that relate the effectiveness of treatments and their associated costs (cost-effectiveness analyses).
JAMAIssues in the Registration of Clinical Trials
218 Citations2007Deborah A. Zarin, Nicholas C. Ide +4 more
Several issues confound the implementation of such a database, including the lack of an accepted format or process for providing summaries of trial results to the public and concerns about disseminating data in the absence of independent scientific review.
ScienceLife Cycle of Translational Research for Medical Interventions
207 Citations2008Despina G. Contopoulos‐Ioannidis, George Α. Alexiou +2 more
From the initial discovery of a medical intervention to a highly cited article is a long road, and even this is not the end of the journey.
BiostatisticsAn invitation to reproducible computational research
173 Citations2010David L. Donoho
The journal Biostatistics makes a formal venture into computational reproducibility, and the policies being adopted are eminently practical, and it is hoped that many authors will begin using this option.
Annals of Internal MedicineDisclosure of Researcher Contributions: A Study of Original Research Articles in <i>The Lancet</i>
171 Citations1999Veronica Yank, Drummond Rennie
increasing microbiological and epidemiologic evidence indicates that H. pylori was once more common, perhaps nearly universal in humans, than it is in the authors' postmodern society and its decline has probably been accelerated by the widespread use of antibiotics in recent decades.
NatureConform and be funded
168 Citations2012Joshua M. Nicholson, John P. A. Ioannidis
Too many US authors of the most innovative and influential papers in the life sciences do not receive NIH funding, contend Joshua M. Nicholson and John P. A. Ioannidis.
PLoS ONEHarmonic Allocation of Authorship Credit: Source-Level Correction of Bibliometric Bias Assures Accurate Publication and Citation Analysis
166 Citations2008Nils T. Hagen
It is shown how harmonic counting, which allocates credit according to authorship rank and the number of coauthors, provides simultaneous source-level correction for both biases as well as accommodating further decoding of byline information.
NatureFund people not projects
158 Citations2011John P. A. Ioannidis
John P. A. Ioannidis proposes ways to save scientists from spending all their time writing grants, as well as ideas to improve the quality of research.
SurgeryRedundant surgical publications: Tip of the iceberg?
157 Citations2001Moshe Schein, Ramesh Paladugu
Almost 1 in every 6 original articles published in leading surgical journals represents some form of redundancy; the phenomenon is widespread, and it cuts across the entire spectrum of surgeons in the United States and abroad.
JAMAClinical Trial Registration
136 Citations2007Christine Lainé, Richard Horton +13 more
This update summarizes that reevaluation of the ICMJE policy on trial registration, specifically commenting on registries that meet the policy requirements, the types of studies that require registration, and the registration of trial results.
Annual Review of Genomics and Human GeneticsThe Power of Meta-Analysis in Genome-Wide Association Studies
129 Citations2013Orestis A. Panagiotou, Cristen J. Willer +2 more
An overview of the current landscape of variants discovered by GWA meta-analyses is provided, and with extrapolations from empirical data the value of larger meta-Analyses for the discovery of additional genetic associations and new biology in the future is assessed.
Science Translational MedicineMaking Prospective Registration of Observational Research a Reality
127 Citations2014Rafael Dal‐Ré, John P. A. Ioannidis +6 more
There is an ethical and scientific imperative to publicly preregister key information from newly approved protocols, which should be required by funders.
Cancer Epidemiology Biomarkers & PreventionTransforming Epidemiology for 21st Century Medicine and Public Health
121 Citations2013Muin J. Khoury, Tram Kim Lam +25 more
Eight overarching thematic recommendations can transform cancer epidemiology and the field of epidemiology, in general, by enhancing transparency, interdisciplinary collaboration, and strategic applications of new technologies.
Journal of Economic SurveysWHAT'S TO KNOW ABOUT THE CREDIBILITY OF EMPIRICAL ECONOMICS?
121 Citations2013John P. A. Ioannidis, Hristos Doucouliagos
European Journal of Clinical InvestigationUndue industry influences that distort healthcare research, strategy, expenditure and practice: a review
114 Citations2013Emmanuel Stamatakis, Richard Weiler +1 more
The enormous financial interests involved in the development and marketing of drugs and devices may have given excessive power to these industries to influence medical research, policy, and practice.
BMJImpact of peer review on reports of randomised trials published in open peer review journals: retrospective before and after study
104 Citations2014Sally Hopewell, Gary S. Collins +7 more
Peer reviewers fail to detect important deficiencies in reporting of the methods and results of randomised trials, and changes requested by peer reviewers was relatively small.
NatureReproducibility: The risks of the replication drive
100 Citations2013Mina J. Bissell
The push to replicate findings could shelve promising research and unfairly damage the reputations of careful, meticulous scientists, says Mina Bissell.
BMJContent area experts as authors: helpful or harmful for systematic reviews and meta-analyses?
92 Citations2012Peter C Gøtzsche, John P. A. Ioannidis
Peter Gøtzsche and John Ioannidis argue that it is not always sensible to include subject experts as authors of systematic reviews and meta-analyses.
JAMAAssessing Value in Biomedical Research
91 Citations2014John P. A. Ioannidis, Muin J. Khoury
The criteria by which scientists and their teams are rewarded for their efforts by agencies that fund them and institutions that host them should be revisited, aligning criteria with the desired outcomes: research that is productive, high-quality, reproducible, shareable, and translatable.
Circulation ResearchPercentile Ranking and Citation Impact of a Large Cohort of National Heart, Lung, and Blood Institute–Funded Cardiovascular R01 Grants
84 Citations2014Narasimhan Danthi, Colin O. Wu +2 more
In a large cohort of NHLBI-funded cardiovascular R01 grants, an observational analysis of percentile rankings and bibliometric outcomes was unable to find a monotonic association between better percentile ranking and higher scientific impact as assessed by citation metrics.
NatureWhy animal research needs to improve
82 Citations2011Malcolm Macleod
Many of the studies that use animals to model human diseases are too small and too prone to bias to be trusted, says Malcolm Macleod.
Trends in Pharmacological SciencesRaw data from clinical trials: within reach?
66 Citations2013Peter Doshi, Steven N. Goodman +1 more
More widespread availability of data from clinical research is argued in favor in order to reduce selective reporting biases and enhance the reproducibility of and trust in clinical research.
PLoS ONEProfit (p)-Index: The Degree to Which Authors Profit from Co-Authors
54 Citations2013N. Ahmad Aziz, Maarten Pieter Rozing
It is shown that the contribution of co-authors to the work of a particular author is generally substantial and that researchers’ relative rankings change materially when adjusted for the contributions ofCo-authors, suggesting that exceptional researchers are characterized by a relatively high degree of scientific independency/originality.
Trends in Molecular MedicineBiospecimens, biomarkers, and burgeoning data: the imperative for more rigorous research standards
49 Citations2012George Poste
A systems approach will require new multi-institution collaborations, the integration of diverse technical and clinical activities, greater engagement of industry, and education of regulators, clinicians, and payers about how to use biomarkers for improved patient management and clinical outcomes.
Archives of Internal MedicineStudy of Neurontin: Titrate to Effect, Profile of Safety (STEPS) Trial
48 Citations2011Samuel D. Krumholz, David Egilman +1 more
The STEPS trial was a seeding trial, used to promote gabapentin and increase prescribing among investigators, and marketing was extensively involved in its planning and implementation.
European Journal of Clinical InvestigationA list of highly influential biomedical researchers, 1996–2011
45 Citations2013Kevin W. Boyack, Richard Klavans +2 more
A list of highly influential biomedical researchers based on Scopus citation data from the period 1996‐2011 is generated, with information on the profile of these most influential authors, including the most common Medical Subject Heading terms in their articles.
American Journal of Public HealthWeight of the Evidence or Wait for the Evidence? Protecting Underground Miners From Diesel Particulate Matter
37 Citations2005Celeste Monforton
This case study profiles a decade-long saga of the Methane Awareness Resource Group Diesel Coalition to impede epidemiological studies on diesel exhaust undertaken by the National Institute for Occupational Safety and Health and the National Cancer Institute.
Genetics in MedicineKnowledge integration at the center of genomic medicine
30 Citations2012Muin J. Khoury, Marta Gwinn +2 more
Three articles in this issue of Genetics in Medicine describe examples of “knowledge integration,” involving methods for generating and synthesizing rapidly emerging information on health-related genomic technologies and engaging stakeholders around the evidence.
Journal of the American Medical Informatics AssociationScientific research in the age of omics: the good, the bad, and the sloppy
21 Citations2012Daniela Witten, Robert Tibshirani
An online, open-access, postpublication, peer review system that will increase the accountability of scientists for the quality of their research and the ability of readers to distinguish good from sloppy science is urged.
JAMAResearch on Peer Review and Biomedical Publication
19 Citations2014Drummond Rennie, Annette Flanagin
This issue of JAMA includes 3 reports1- 3 first presented at the Seventh International Congress on Peer Review and Biomedical Publication in September 2013, which substantially broadened to include all aspects of biomedical publication.
The FASEB JournalIs there a glass ceiling for highly cited scientists at the top of research universities?
18 Citations2010John P. A. Ioannidis