Measuring research excellence

It is found that conventional bibliometric indicators, numbers of publications, citations, and top 1 per cent most cited publications, correlate with the number of Nobel Prize achievements in several advanced countries with similar research abilities and their use as indicators of research excellence is not valid.

Abstract

Purpose Several bibliometric indicators that are extensively used to estimate research performance have not been validated against an external criterion of excellence. This paper aims to investigate whether this validation is possible using the number of Nobel Prize awards. Design/methodology/approach This study uses several analytical treatments of the data to investigate: whether Nobel Prize awards are sporadic events or they depend on the scientific activity of countries or institutions and can be used in research evaluation; and the association between the number of Nobel Prize achievements and conventional bibliometric indicators across countries and institutions. Findings This study finds that conventional bibliometric indicators, numbers of publications, citations, and top 1 per cent most cited publications, correlate with the number of Nobel Prize achievements in several advanced countries with similar research abilities. Contrarily, in countries and institutions with more variable research characteristics, there is no association between conventional bibliometric indicators and the number of Nobel Prize achievements, and their use as indicators of research excellence is not valid. In contrast, the number of national articles in Nature and Science correlates with the number of Nobel Prize achievements across countries and institutions. Practical implications Science administrators implementing research evaluations and research incentives based on conventional bibliometric indicators should consider that increasing the scores of these indicators does not imply an improvement in research excellence. Originality/value The study demonstrates that Nobel Prize achievements are not singular events that occur by chance. Therefore, the number of Nobel Prize achievements can be used to validate bibliometric indicators.