What lies behind the averages and significance of citation indicators in different disciplines?

It is shown that the most popular journal citation indicators used in quality assessment — the journal impact factors of Thomson Scientific and the scientific journal rankings of Scopus — are strongly correlated with the proportion of within-database references, and even more so with the number ofWithin-database recent references per paper.

Abstract

The limitations of citation-based indicators include a lack of coverage, no normalization with respect to the length of reference lists (with a potential bias in favour of reviews), and different citation habits. As a consequence, the distributions of the indicators are not comparable across different disciplines. Here we show that the most popular journal citation indicators used in quality assessment — the journal impact factors of Thomson Scientific and the scientific journal rankings of Scopus — are strongly correlated with the proportion of within-database references, and even more so with the number of within-database recent references per paper. No significant correlations were found with other bibliometric magnitudes. We anticipate that these results will be a starting point for more sophisticated indicator models that take this dependence into account, and for the design of strategies aimed at extending such bibliometric databases as Thomson Scientific’s Science Citation Index or Elsevier’s Scopus to improve their capacity to evaluate all sciences.