The inconsistency of the h‐index
Journal of the American Society for Information Science and TechnologyPublished 31 October 2011
Ludo Waltman, Nees Jan van Eck
Citations333
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
It is argued that for the purpose of measuring the overall scientific impact of a scientist (or some other unit of analysis), the h-index behaves in a counterintuitive way and cannot be considered an appropriate indicator of a Scientist's overall scientificimpact.
Abstract
Abstract The h‐index is a popular bibliometric indicator for assessing individual scientists. We criticize the h‐index from a theoretical point of view. We argue that for the purpose of measuring the overall scientific impact of a scientist (or some other unit of analysis), the h‐index behaves in a counterintuitive way. In certain cases, the mechanism used by the h‐index to aggregate publication and citation statistics into a single number leads to inconsistencies in the way in which scientists are ranked. Our conclusion is that the h‐index cannot be considered an appropriate indicator of a scientist's overall scientific impact. Based on recent theoretical insights, we discuss what kind of indicators can be used as an alternative to the h‐index. We pay special attention to the highly cited publications indicator. This indicator has a lot in common with the h‐index, but unlike the h‐index it does not produce inconsistent rankings.
Keywords
Decision SciencesArts and Humanities
Proceedings of the National Academy of SciencesAn index to quantify an individual's scientific research output
11,535 Citations2005J. E. Hirsch
The index hbar, defined as the number of papers of an individual that have citation count larger than or equal to the citation count of all coauthors of each paper, is proposed as a useful index to characterize the scientific output of a researcher that takes into account the effect of multiple authorship.
ScientometricsTheory and practise of the g-index
2,302 Citations2006Leo Egghe
It is shown that the g-index inherits all the good properties of the h-index and better takes into account the citation scores of the top articles and yields a better distinction between and order of the scientists from the point of view of visibility.
Proceedings of the National Academy of SciencesDoes the <i>h</i> index have predictive power?
968 Citations2007J. E. Hirsch
The findings indicate that the h index is better than other indicators considered (total citation count, citations per paper, and total paper count) in predicting future scientific achievement.
Journal of Informetricsh-Index: A review focused in its variants, computation and standardization for different scientific fields
833 Citations2009Sergio Alonso, Francisco Javier Cabrerizo +2 more
This contribution presents a comprehensive review on the h-index and related indicators field, studying their main advantages, drawbacks and the main applications that can be found in the literature.
ScientometricsA Hirsch-type index for journals
670 Citations2006Tibor Braun, Wolfgang Glänzel +1 more
It is suggested that a h-type index - equal to h if you have published h papers, each of which has at least h citations - would be a useful supplement to journal impact factors.
Journal of the American Society for Information Science and TechnologyWhat do we know about the <i>h</i> index?
663 Citations2007Lutz Bornmann, Hans‐Dieter Daniel
The advantages and disadvantages of the h index are described and the studies on the convergent validity of this index are summarized and corrections and complements as well as single-number alternatives are introduced.
ScientometricsComparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups
609 Citations2006Anthony F. J. van Raan
Characteristics of the statistical correlation between the Hirsch (h-) index and several standard bibliometric indicators are presented, as well as with the results of peer review judgment, which show that the h-index and the bibliometry ‘crown indicator’ both relate in a quite comparable way with peer judgments.
Journal of InformetricsThe h-index: Advantages, limitations and its relation with other bibliometric indicators at the micro level
493 Citations2007Rodrigo Costas, María Bordons
The relationship of the h-index with other bibliometric indicators at the micro level is analysed for Spanish CSIC scientists in Natural Resources, using publications downloaded from the Web of Science (1994–2004).
Journal of InformetricsTowards a new crown indicator: Some theoretical considerations
422 Citations2010Ludo Waltman, Nees Jan van Eck +3 more
This work critically examine the theoretical basis of the normalization mechanism applied in the crown indicator and makes a comparison with an alternativenormalization mechanism, which turns out to have more satisfactory properties than the mechanism applied.
ScientometricsAn index to quantify an individual’s scientific research output that takes into account the effect of multiple coauthorship
408 Citations2010J. E. Hirsch
EconometricaThe Measurement of Intellectual Influence
323 Citations2004Ignacio Palacios‐Huerta, Oscar Volij
ScientometricsAn informetric model for the Hirsch-index
308 Citations2006Leo Egghe, Ronald Rousseau
It is shown that in each practical situation an IPP always has a unique h-index, which is based on the total number of items in Lotkaian systems.
ScientometricsOn the h-index - A mathematical approach to a new measure of publication activity and citation impact
306 Citations2006Wolfgang Glänzel
L'index-h, fonde sur le nombre de citations recues, mesure l'activite de publication and l'impact en citations, c'est un indicateur utile avec d'interessantes proprietes mathematiques, mais qui ne saurait se substituer aux indicateurs bibliometriques courants plus sophistiques.
Journal of InformetricsA multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants
289 Citations2011Lutz Bornmann, Rüdiger Mutz +2 more
NatureMeasures for measures
283 Citations2006Sune Lehmann, A.D. Jackson +1 more
Comparing commonly used measures of author quality, the mean number of citations per paper emerges as a better indicator than the more complex Hirsch index; a third method, the number of papers published per year, measures industry rather than ability.
Journal of InformetricsLifting the crown—citation z-score
273 Citations2007Jonas Lundberg
An item oriented field normalized citation score average (c¯f) is an incremental improvement as it differs from the crown indicator in so much as normalization takes place on the level of individual publication (or item) rather than on aggregated levels, and therefore assigns equal weight to each publication.
ScientometricsBenchmarking international scientific excellence: Are highly cited research papers an appropriate frame of reference?
254 Citations2002Robert Tijssen, Martijn S. Visser +1 more
A citation-based "systems approach" for analyzing the various institutional and cognitive dimensions of scientific excellence within national research systems, focusing on the most highly cited research papers in the international journal literature, indicates that these high performance papers provide a useful analytical framework.
Journal of InformetricsA modification of the h-index: The hm-index accounts for multi-authored manuscripts
222 Citations2008Michael Schreiber
The effect of this procedure is compared with other variants of the Hirsch index and found to be superior to the fractionalised counting of citations and to the normalization of the h-index with the average number of authors in theh-core.
Data Archiving and Networked Services (DANS)Generalizing the h- and g-indices
163 Citations2008Nees Jan van Eck, Ludo Waltman
Mathematical Social SciencesAn axiomatic characterization of the Hirsch-index
147 Citations2008Gerhard J. Woeginger
An axiomatic characterization of the Hirsch-index in terms of three natural axioms is presented, which is a well-known index for measuring and comparing the output of scientific researchers.
Journal of InformetricsHirsch's h-index: A stochastic model
138 Citations2006Quentin L. Burrell
It is conjecture that the h-index is, according to this model, (approximately) linear in career length, log publication rate and log citation rate, at least for moderate citation rates.
Journal of the American Society for Information Science and TechnologyMathematical theory of the h‐ and g‐index in case of fractional counting of authorship
135 Citations2008Leo Egghe
This article studies the h-index (Hirsch index) and the g-index of authors, in case one counts authorship of the cited articles in a fractional way.
Journal of the American Society for Information Science and TechnologyThe w‐index: A measure to assess scientific impact by focusing on widely cited papers
115 Citations2009Qiang Wu
Journal of DocumentationThe <i>h</i>‐index: a broad review of a new bibliometric indicator
112 Citations2010Michael Norris, Charles Oppenheim
How the h‐index has become a subject of widespread debate, how it has spawned many variants and diverse applications since first introduced in 2005 and some of the issues in its use are shown.
Journal of the American Society for Information Science and TechnologyRanking scientists and departments in a consistent manner
110 Citations2011Denis Bouyssou, Thierry Marchant
The consequences of consistency are explored and two families of consistent rankings are characterized, both of which satisfy a property called consistency, recently introduced in the literature.
ScientometricsAn axiomatic characterization of the ranking based on the h-index and some other bibliometric rankings of authors
103 Citations2009Thierry Marchant
A formal framework in which rankings can be axiomatically characterized is proposed and it is argued that such analyses can help the user of a ranking to choose one that is adequate in the context where she/he is working.
ScientometricsA quantitative analysis of indicators of scientific performance
99 Citations2008Sune Lehmann, A.D. Jackson +1 more
ScientometricsThe significance of the number of highly cited papers as an indicator of scientific prolificacy
72 Citations1990R. Plomp
An analysis of the distribution of HCPs over the years provides support for the claim that the number of highly cited papers has some advantages as an indicator of an author's scientific impact.
Journal of InformetricsBibliometric rankings of journals based on Impact Factors: An axiomatic approach
71 Citations2010Denis Bouyssou, Thierry Marchant
An axiomatic analysis of Impact Factors when used as tools for ranking journals draws on the similarities between the problem of comparing distribution of citations among papers and that of comparing probability distributions on consequences as commonly done in decision theory.
Annalen der PhysikTwenty Hirsch index variants and other indicators giving more or less preference to highly cited papers
68 Citations2010Michael Schreiber
I analyse the citation records of 26 physicists discussing various suggestions of Hirsch index, quantifying which indices and indicators yield similar and which yield more deviating rankings of the 26 datasets.
ScientometricsThe highly cited papers of professors as an indicator of a research group's scientific performance
54 Citations1994R. Plomp
The number of HCPs scored by the professors (and other senior scientists) during their entire career is a much more reliable predictor of the performance of a research group than the number of short-term citations of the articles published by the group within a short period.
Journal of the American Society for Information Science and TechnologyScore‐based bibliometric rankings of authors
53 Citations2009Thierry Marchant
An axiomatic analysis of the family of all scoring rules and of some particular cases within this family of authors is presented.
ScientometricsOn measuring scholarly influence by citations
53 Citations2011Martin Ravallion, Adam Wagstaff
The paper proposes a new approach that is grounded on a theoretical “influence function,” representing explicit prior beliefs about how citations reflect influence, and conditions are derived for robust qualitative comparisons of influence.
National Bureau of Economic ResearchHow Does the Market Use Citation Data? The Hirsch Index in Economics
47 Citations2010Glenn Ellison
Journal of InformetricsThe diffusion of H-related literature
40 Citations2011Lin Zhang, Bart Thijs +1 more
The present study attempts to trace the diffusion of h-related literature over a five-year period beginning with the introduction of the h-index using a reliable and representative publication set of 755 papers retrieved from the Web of Science database.
Journal of the American Society for Information Science and TechnologyThe relation between Eigenfactor, audience factor, and influence weight
36 Citations2010Ludo Waltman, Nees Jan van Eck
A theoretical and empirical analysis of a number of bibliometric indicators of journal performance, namely the Eigenfactor indicator, the audience factor, and the influence weight indicator finds that the last two indicators can be regarded as a kind of special cases of the first indicator.
Journal of InformetricsThe measurement of low- and high-impact in citation distributions: Technical results
36 Citations2010Pedro Albarrán, Ignacio Ortuño‐Ortín +1 more
A novel methodology for comparing the citation distributions of research units of a certain size working in the same homogeneous field is introduced, using two real valued indicators to describe the shape of any distribution: a high-impact and a low-impact measure defined over the set of articles with citations above or below the CCL.
ScientometricsMore axiomatics for the Hirsch index
31 Citations2009Antonio Quesada
This note suggests three characterizations of the Hirsch index using monotonicity as one of the axioms without adopting themonotonicity axiom.
ScientometricsA new family of scientific impact measures: The generalized Kosmulski-indices
28 Citations2009Vladimir G. Deı̌neko, Gerhard J. Woeginger
The generalized Kosmulski-indices are introduced as a new family of scientific impact measures for ranking the output of scientific researchers and the main contribution is an axiomatic characterization that characterizes every generalized Kosmoski-index in terms of three axioms.
Journal of InformetricsCharacterizations of the generalized Wu- and Kosmulski-indices in Lotkaian systems
16 Citations2011Leo Egghe
The generalized Wu- and Kosmulski-indices are defined, allowing for general parameters of multiplication or exponentiation, and formulae for these generalized indices are presented in a Lotkaian framework.
RePub (Erasmus University, Rotterdam)A Taxonomy of Bibliometric Performance Indicators Based on the Property of Consistency
13 Citations2009Ludo Waltman, Nees Jan van Eck
The h-index is shown not to have this important property of consistency, and a taxonomy of bibliometric indicators of scientific performance is proposed.