<i>Caveats</i> for the use of citation indicators in research and journal evaluations
Journal of the American Society for Information Science and TechnologyPublished 6 November 2007Open access
Loet Leydesdorff
Citations193
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The assumption that citation and publication practices are homogenous within specialties and fields of science is invalid and the delineation of fields and among specialties is fuzzy.
Abstract
Abstract Aging of publications, percentage of self‐citations, and impact vary from journal to journal within fields of science. The assumption that citation and publication practices are homogenous within specialties and fields of science is invalid. Furthermore, the delineation of fields and among specialties is fuzzy. Institutional units of analysis and persons may move between fields or span different specialties. The match between the citation index and institutional profiles varies among institutional units and nations. The respective matches may heavily affect the representation of the units. Non‐Institute of Scientific Information (ISI) journals are increasingly cornered into “transdisciplinary” Mode‐2 functions with the exception of specialist journals publishing in languages other than English. An “externally cited impact factor” can be calculated for these journals. The citation impact of non‐ISI journals will be demonstrated using Science and Public Policy as the example.
Keywords
Decision Sciences
Proceedings of the National Academy of SciencesAn index to quantify an individual's scientific research output
11,535 Citations2005J. E. Hirsch
The index hbar, defined as the number of papers of an individual that have citation count larger than or equal to the citation count of all coauthors of each paper, is proposed as a useful index to characterize the scientific output of a researcher that takes into account the effect of multiple authorship.
SociometryA Set of Measures of Centrality Based on Betweenness
10,249 Citations1977Linton C. Freeman
A family of new measures of point and graph centrality based on early intuitions of Bavelas (1948) is introduced, used to index centrality in any large or small network of symmetrical relations, whether connected or unconnected.
Choice Reviews OnlineThe New production of knowledge: the dynamics of science and research in contemporary societies
7,100 Citations1995
Contemporary Sociology A Journal of ReviewsRe-Thinking Science: Knowledge and the Public in an Age of Uncertainty
3,413 Citations2003Fletcher Linder, Joe Spear +3 more
ScienceCitation Analysis as a Tool in Journal Evaluation
2,760 Citations1972Eugene Garfield
In 1971, the Institute for Scientfic Information decided to undertake a systematic analysis of journal citation patterns across the whole of science and technology.
BMJWhy the impact factor of journals should not be used for evaluating research
2,448 Citations1997Per O. Seglen
Technology and CultureCitation Indexing-Its Theory and Application in Science, Technology, and Humanities
1,669 Citations1980Jack Goodwin, Eugene Garfield
Citation indexing-its theory and application in science, technology, and humanities, Citation indexing (Citation Indexing) (CIFS), مرکز فناوری اطلاعات (Citations Indexing),
Information science and knowledge managementCitation Analysis in Research Evaluation
1,319 Citations2005Henk F. Moed
This work focuses on assessing Basic Science Research Departments and Scientific Journals, as well as Empirical and Theoretical Chapters, and the Citation Indexes, which summarize the literature on empirical and theoretical determinants of scientific research.
The Canadian Journal of SociologyThe Intellectual and Social Organization of the Sciences
1,246 Citations1986Bernard Barber, Richard Whitley
This chapter discusses reputational control over Scientific Work and the growth of Employment Opportunities for Scientists and the Degree of Mutual Dependence between scientists and the Organization of Scientific Fields.
NatureThe scientific impact of nations
1,224 Citations2004David A. King
What different countries get for their research spending is illustrated in the chart below, which shows how research spending in these countries has changed over the past 50 years.
The Intellectual and Social Organization Of the Sciences
1,050 Citations2000Richard Whitley
Requisite Variety and Its Implications for the Control of Complex Systems
970 Citations1991W. Ross Ashby
This paper proposes to show the relationship between the two theorems of Shannon on the quantity of noise or error that could be removed through a correction-channel, and to indicate something of their implications for regulation, in the cybernetic sense, when the system to be regulated is extremely complex.
ScientometricsMapping the backbone of science
952 Citations2005Kevin W. Boyack, Richard Klavans +1 more
A new map representing the structure of all of science, based on journal articles, is presented, including both the natural and social sciences, including biochemistry, which appears as the most interdisciplinary discipline in science.
Journal of the American Society for Information Science and TechnologyRequirements for a cocitation similarity measure, with special reference to Pearson's correlation coefficient
807 Citations2003Per Ahlgren, Bo Jarneving +1 more
It is concluded that Pearson's r is probably not an optimal choice of a similarity measure in ACA and further empirical research is needed to show if, and in that case to what extent, the use of similarity measures in ACA that fulfill these requirements would lead to objectively better results In full-scale studies.
Information Processing & ManagementAn introduction to informetrics
756 Citations1992Jean Tague‐Sutcliffe
The scope and significance of the field of informetrics is defined and related to the earlier fields of bibliometrics and scientometrics; the phenomena studied by informetricians are identified.
Journal of Information Science"Sources of information on specific subjects" by S.C. Bradford
703 Citations1985B. C. Brookes
ScientometricsA Hirsch-type index for journals
670 Citations2006Tibor Braun, Wolfgang Glänzel +1 more
It is suggested that a h-type index - equal to h if you have published h papers, each of which has at least h citations - would be a useful supplement to journal impact factors.
Journal of the American Society for Information Science and TechnologyBetweenness centrality as an indicator of the interdisciplinarity of scientific journals
617 Citations2007Loet Leydesdorff
Betweenness centrality is shown to be an indicator of the interdisciplinarity of journals, but only in local citation environments and after normalization; otherwise, the influence of degree centrality (size) overshadows the betweenness-centrality measure.
Information Processing & ManagementCitation influence for journal aggregates of scientific publications: Theory, with application to the literature of physics
609 Citations1976Gabriel Pinski, Francis Narin
A self-consistent methodology is developed for determining citation based influence measures for scientific journals, subfields and fields and a wide range in influence is found between the most influential and least influential or peripheral journals.
Chinese Science BulletinThe R- and AR-indices: Complementing the h-index
541 Citations2007Bihui Jin, Liming Liang +2 more
The R- and AR-indices are introduced and it is proposed the pair (h, AR) as a meaningful indicator for research evaluation and a relation characterizing the h-index in the power law model is proved.
BMJFortnightly Review: How can impact factors be improved?
485 Citations1996Eugene Garfield
Fourteen year cumulative impact data for 10 leading medical journals provide a quantitative indicator of their long term influence.
Research PolicyThe use of bibliometric data for the measurement of university research performance
482 Citations1985Henk F. Moed, W. J. M. Burger +2 more
ScientometricsThe difficulty of achieving full coverage of international social science literature and the bibliometric consequences
387 Citations1999Diana Hicks
The paper examines the main reasons why social science bibliometrics are problematic, namely: the centrality of books in social science literature and their high citation rate; and the national orientation of social science literatures.
ScientometricsA new classification scheme of science fields and subfields designed for scientometric evaluation purposes
337 Citations2003Wolfgang Glänzel, A. Schubert
A two-level hierarchic system of fields and subfields of the sciences, social sciences and arts and humanities is proposed, specifically designed for scientometric purposes with the ultimate goal of classifying every single document into a well-defined category.
ScientometricsTheories of citation?
333 Citations1998Loet Leydesdorff
The specific contexts represented in the modern citation can be deconstructed from the perspective of the cultural evolution of scientific communication.
CERN Document Server (European Organization for Nuclear Research)The Knowledge-Based Economy: Modeled, Measured, Simulated
240 Citations2006Loet Leydesdorff
ScientometricsIs the United States losing ground in science? A global perspective on the world science system
205 Citations2008Loet Leydesdorff, Caroline S. Wagner
Based on the Science Citation Index-Expanded web-version, the USA is still by far the strongest nation in terms of scientific performance but its relative decline in percentage share of publications is largely due to the emergence of China and other Asian nations.
Evolutionary Economics and Chaos Theory: New Directions in Technology Studies
178 Citations1994Loet Leydesdorff, Peter van den Besselaar
ScientometricsAssessing the usefulness of bibliometric indicators for the humanities and the social and beha vioural sciences: A comparative study
154 Citations1989A. J. Nederhof, Rolf A. Zwaan +2 more
Although many publications were written in English, only experimental psychology, general linguistics, anthropology, and genrral literature were internationally oriented regarding output media.
ScientometricsAre the contributions of China and Korea upsetting the world system of science?
144 Citations2005Loet Leydesdorff, Zhou Ping
Institutions and their aggregates are not the right units of analysis for developing a science policy with cognitive goals in view; institutions, however, can be compared in terms of their performance with reference to their previous stages.
ScientometricsHow cross-disciplinary is bionanotechnology? Explorations in the specialty of molecular motors
143 Citations2007Ismael Ràfols, Martin Meyer
It is found that there is a consistent high degree of cross-disciplinarity in the cognitive practices of research but a more erratic and narrower degree in the social dimensions.
arXiv (Cornell University)A New Era in Citation and Bibliometric Analyses: Web of Science, Scopus, and Google Scholar
140 Citations2006Lokman I. Meho, Kiduk Yang
The strengths and weaknesses of WoS, Scopus, and GS, their overlap and uniqueness, quality and language of the citations, and the implications of the findings for citation analysis are discussed.
Journal of the American Society for Information Science and TechnologyCan scientific journals be classified in terms of aggregated journal‐journal citation relations using the <i>Journal Citation Reports</i>?
136 Citations2006Loet Leydesdorff
The aggregated citation relations among journals included in the Science Citation Index provide us with a huge matrix which can be analyzed in various ways and the factor-analytic solutions allow us to test classifications against the structures contained in the database.
ScientometricsA comparison of the knowledge-based innovation systems in the economies of South Korea and the Netherlands using Triple Helix indicators
135 Citations2005Han Woo Park, Heung Deug Hong +1 more
The Triple Helix model for measuring the emergence of a knowledge base of socio-economic systems is elaborates on, using multiple indicators: webometric, scientometric, and technometric to examine the current state of the innovation systems of South Korea and the Netherlands.
Science Technology & Human ValuesDimensions of Citation Analysis
131 Citations1990Loet Leydesdorff, Olga Amsterdamska
ScientometricsHirsch index or Hirsch rate? Some thoughts arising from Liang’s data
95 Citations2007Quentin L. Burrell
This work re-analyse a previously published data set which reveals that in many cases the h-rate, according to theory, is not constant, and suggests ways in which deeper scientometric investigations could be carried out.
Journal of the American Society for Information Science and TechnologyVisualization of the citation impact environments of scientific journals: An online mapping exercise
81 Citations2006Loet Leydesdorff
Aggregated journal-journal citation networks based on the Journal Citation Reports 2004 of the Science Citation Index and the Social Science Citation Index are made accessible from the perspective of any of these journals, and the user is able to analyze the citation environment in terms of links and graphs.
ScientometricsSelf-citations, co-authorships and keywords: A new approach to scientists’ field mobility?
81 Citations2007Iina Hellsten, Renaud Lambiotte +2 more
It is shown that author’s self-citation patterns reveal important information on the development and emergence of new research topics over time, and a network based definition of field mobility is introduced, using the Optimal Percolation Method.
Social Science InformationFilling the gap between discussions on science and scientists' everyday activities: applying the autopoiesis system theory to scientific knowledge
80 Citations1998Yuko Fujigaki
ScientometricsMapping interdisciplinarity at the interfaces between the Science Citation Index and the Social Science Citation Index
75 Citations2007Loet Leydesdorff
On the basis of the resulting visualizations, “betweenness” — a measure from social network analysis — is suggested as an indicator for measuring the interdisciplinarity of journals.
Journal of the American Society for Information Science and TechnologyArticle impact calculated over arbitrary periods
69 Citations2004Tove Faber Frandsen, Ronald Rousseau
This paper addresses the various formulations of impact of articles, usually groups of articles as gauged by citations that these articles receive over a certain period of time, and shows how an impact factor for the general case can be defined.
ScientometricsScientometrics and communication theory: Towards theoretically informed indicators
66 Citations1997Loet Leydesdorff, Peter van den Besselaar
ScientometricsApplied scientometrics: An assessment methodology for developing countries
63 Citations1985Michael J. Moravcsik
A United Nations sponsored project is described to formulate a practicable method for assessing the impact of science and technology in the developing countries and to propose further research to improve the development of such indicators.
Journal of the American Society for Information Science and TechnologyMapping the <i>Chinese Science Citation Database</i> in terms of aggregated journal–journal citation relations
51 Citations2005Loet Leydesdorff, Bihui Jin
Methods developed for mapping the journal structures contained in aggregated journal-journal citations in the Science Citation Index are applied to the Chinese Science Citation Database of the Chinese Academy of Sciences and it is shown that this data is dually structured.
Library Resources and Technical ServicesScientific and Technical Serials Holdings Optimization in an Inefficient Market: A LSU Serials Redesign Project Exercise
46 Citations1998Stephen J. Bensman, Stanley Wilder
The hypothesis that human knowledge functions on the same probability structure is biological nature und society is developed, and it is stated that librarians will have to change the nature of this system by utilizing the new technology’s capability of delivering information rapidly to move from subscriptions to a free market in ST information through document delivery.
New Tools for Improving and Evaluating The Effectiveness of Research
44 Citations1983Irving H. Sher, Eugene Garfield
The availability of these services now makes practicable several techniques, highly valuable in helping to assist projects from initial conception of an idea through utilization of the results after publication, and these techniques also provide new facta for use in evaluating the work of individuals and organizations.
ScientometricsOnline determination of the journal impact factor and its international properties
43 Citations1997F. Hjortgaard Christensen, Peter Ingwersen +1 more
The method is illustrated by calculating the generalised JIF, self-citations and ECIF(isi) as well as the international impact for Journal of Documentation and Scientometrics.
E-LIS Repository (University of Naples Federico II)Simple models and the corresponding h- and g-index
42 Citations2006Ronald Rousseau
Journal of the American Society for Information Science and TechnologyA comparison between the <i>China Scientific and Technical Papers and Citations Database</i> and the <i>Science Citation Index</i> in terms of journal hierarchies and interjournal citation relations
34 Citations2006Zhou Ping, Loet Leydesdorff
Science and Public PolicyOn-line approaches to measuring national scientific output: a cautionary tale
31 Citations1988Francis Narin, Kimberly Stevens +5 more
CHIMIA International Journal for ChemistryCitation Environment of Angewandte Chemie
29 Citations2007Lutz Bornmann, Loet Leydesdorff +1 more
The environment analyses give a detailed insight into the field-embeddedness of Angewandte Chemie and gives an answer to the question what the local impact of this and other journals in the environment is.
ScientometricsThe citation impacts and citation environments of Chinese journals in mathematics
27 Citations2007Zhou Ping, Loet Leydesdorff
Citation impacts of three Chinese journals in mathematics are analyzed and local citation impacts reflect a journal’s status and function better than (global) impact factors and authors in Chinese journals prefer international instead of domestic ones as sources for their citations.
Quality & QuantityThe See-Saw Effect: a multilevel problem?
25 Citations1988G. G. Kreft, Edith D. de Leeuw
Journal of DocumentationBuilding a list of journals with constructed impact factors
24 Citations1999Johannes Stegmann
The online retrieval from the host DIMDI of the data needed for impact factor calculation is described in detail and the possible usefulness of constructed impact factors for citation and evaluation studies is discussed.
Angewandte Chemie International EditionAngewandte Chemie in Light of the Science Citation Index
22 Citations2001Werner Marx
ScientometricsOn generalising scientometric journal mapping beyond ISI's journal and citation databases
21 Citations1995Robert Tijssen, Thed N. van Leeuwen
Results of this study indicate that a journal content-based map was not only far superior to the journal citation map, but also outperformed the map derived from the combination of both types of data.
ScientometricsEverything about genes: Some results on the dynamics of genomics research
10 Citations2008Robert R. Braam
This study introduces a general model of research dynamics and timing of research management, building on ideas of Price and Bonaccorsi, and applies this model to Research Centers of the Netherlands Genomics Initiative (NGI) and discusses findings.
Journal of the American Society for Information Science and TechnologyTimelines in citation research
10 Citations2006Ronald Rousseau
The timeline used in ISI's Journal Citation Reports (JCR; Thomson ISI, formerly the Institute for Scientific Information, Philadelphia, PA) for half-life calculations, is not a timeline for (average) cited age.
World Journal of GastroenterologyHow to establish a first-class international scientific journal in China?
4 Citations2006Zhen-Xi Li
It is recommended that the effective resolution to the above issue is to learn from Chinese examples such as WJG rather than from "how foreign journals do".
arXiv (Cornell University)Korean journals in the Science Citation Index: What do they reveal about the intellectual structure of S&T in Korea?
3 Citations2009Han Woo Park, Loet Leydesdorff