Clustering scientific documents with topic modeling
ScientometricsPublished 5 May 2014
Chyi-Kwei Yau, Alan L. Porter, Nils C. Newman, Arho Suominen
Citations229
SJR quartileQ1
SJR score1.15
SNIP1.82
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper investigates methods, including LDA and its extensions, for separating a set of scientific publications into several clusters and explores potential scientometric applications of such text analysis capabilities.
Abstract
Abstract Topic modeling is a type of statistical model for discovering the latent “topics” that occur in a collection of documents through machine learning. Currently, latent Dirichlet allocation (LDA) is a popular and common modeling approach. In this paper, we investigate methods, including LDA and its extensions, for separating a set of scientific publications into several clusters. To evaluate the results, we generate a collection of documents that contain academic papers from several different fields and see whether papers in the same field will be clustered together. We explore potential scientometric applications of such text analysis capabilities.
Keywords
Computer ScienceBiochemistry, Genetics and Molecular Biology
Journal of Machine Learning ResearchLatent dirichlet allocation
27,049 Citations2003David M. Blei, Andrew Y. Ng +1 more
Proceedings of the National Academy of SciencesFinding scientific topics
6,017 Citations2004Thomas L. Griffiths, Mark Steyvers
A generative model for documents is described, introduced by Blei, Ng, and Jordan, and a Markov chain Monte Carlo algorithm is presented for inference in this model, which is used to analyze abstracts from PNAS by using Bayesian model selection to establish the number of topics.
Probabilistic latent semantic indexing
3,916 Citations1999Thomas Hofmann
Probabilistic Latent Semantic Indexing is a novel approach to automated document indexing which is based on a statistical latent class model for factor analysis of count data.
Journal of the American Statistical AssociationHierarchical Dirichlet Processes
3,567 Citations2006Yee Whye Teh, Michael I. Jordan +2 more
This work considers problems involving groups of data where each observation within a group is a draw from a mixture model and where it is desirable to share mixture components between groups, and considers a hierarchical model, specifically one in which the base measure for the childDirichlet processes is itself distributed according to a Dirichlet process.
Dynamic topic models
2,322 Citations2006David M. Blei, John Lafferty
A family of probabilistic time series models is developed to analyze the time evolution of topics in large document collections, and dynamic topic models provide a qualitative window into the contents of a large document collection.
Choice Reviews OnlineThe text mining handbook: advanced approaches in analyzing unstructured data
1,626 Citations2007
OPAL (Open@LaTrobe) (La Trobe University)A correlated topic model of Science
1,137 Citations2018David M. Blei, John Lafferty
Annual Review of Information Science and TechnologyScholarly communication and bibliometrics
1,124 Citations2002Christine L. Borgman, Jonathan Furner
The Future of Bibliometrics and its Applications: A Case Study of Agricultural Research Within the European Community Core Journals of the Rapidly Changing Research Front of "Superconductivity" is reviewed.
Journal of Statistical Software<b>topicmodels</b>: An<i>R</i>Package for Fitting Topic Models
1,088 Citations2011Bettina Grün, Kurt Hornik
The R package topicmodels provides basic infrastructure for fitting topic models based on data structures from the text mining package tm to estimate the similarity between documents as well as between a set of specified keywords using an additional layer of latent variables.
LDA-based document models for ad-hoc retrieval
1,086 Citations2006Xing Wei, W. Bruce Croft
This paper proposes an LDA-based document model within the language modeling framework, and evaluates it on several TREC collections, and shows that improvements over retrieval using cluster-based models can be obtained with reasonable efficiency.
Technological Forecasting and Social ChangeForecasting emerging technologies: Use of bibliometrics and patent analysis
1,080 Citations2006Tuğrul Daim, Guillermo Rueda +2 more
The forecasts for three emerging technology areas are presented by integrating the use of bibliometrics and patent analysis into well-known technology forecasting tools such as scenario planning, growth curves and analogies.
Topic modeling
1,057 Citations2006Hanna Wallach
A hierarchical generative probabilistic model that incorporates both n-gram statistics and latent topic variables by extending a unigram topic model to include properties of a hierarchical Dirichlet bigram language model is explored.
Neural Information Processing SystemsHierarchical Topic Models and the Nested Chinese Restaurant Process
938 Citations2003Thomas L. Griffiths, Michael I. Jordan +2 more
A Bayesian approach is taken to generate an appropriate prior via a distribution on partitions that allows arbitrarily large branching factors and readily accommodates growing data collections.
Neural Information Processing SystemsCorrelated Topic Models
929 Citations2005John Lafferty, David M. Blei
The correlated topic model (CTM) is developed, where the topic proportions exhibit correlation via the logistic normal distribution and a mean-field variational inference algorithm is derived for approximate posterior inference in this model, which is complicated by the fact that the Logistic normal is not conjugate to the multinomial.
The Annals of Applied StatisticsA correlated topic model of Science
921 Citations2007David M. Blei, John D. Lafferty
The correlated topic model (CTM) is developed, where the topic proportions exhibit correlation via the logistic normal distribution, and it is demonstrated its use as an exploratory tool of large document collections.
Technological Forecasting and Social ChangeDiffusion of nuclear energy in some developing countries
705 Citations2013Alessandra Dalla Valle, Claudia Furlan
Cambridge University Press eBooksThe Text Mining Handbook
665 Citations2006Ronen Feldman, James R. Sanger
Probabilistic author-topic models for information discovery
581 Citations2004Mark Steyvers, Padhraic Smyth +2 more
The methodology is applied to a large corpus of 160,000 abstracts and 85,000 authors from the well-known CiteSeer digital library, and a model with 300 topics is learned using a Markov chain Monte Carlo algorithm.
Proceedings of the VLDB EndowmentAn architecture for parallel topic models
431 Citations2010Alexander J. Smola, Shravan Narayanamurthy
This paper describes a high performance sampling architecture for inference of latent topic models on a cluster of workstations and shows that this architecture is entirely general and that it can be extended easily to more sophisticated latent variable models such as n-grams and hierarchies.
Text Mining
386 Citations2009
This book examines methods to automatically cluster and classify text documents and applies these methods in a variety of areas, including adaptive information filtering, information distillation, and text search.
Text Mining Handbook: Advanced Approaches in Analyzing Unstructured Data
201 Citations2006Ronen Feldman, James R. Sanger
Lecture notes in computer sciencePLDA: Parallel Latent Dirichlet Allocation for Large-Scale Applications
190 Citations2009Yi Wang, Hongjie Bai +3 more
PLDA smooths out storage and computation bottlenecks and provides fault recovery for lengthy distributed computations and can be applied to large, real-world applications and achieves good scalability.
Technological Forecasting and Social Change“Term clumping” for technical intelligence: A case study on dye-sensitized solar cells
157 Citations2014Yi Zhang, Alan L. Porter +3 more
Six "term clumping" steps that can clean and consolidate topical content in text sources and are better suited to contributing usable technical intelligence than the original results are presented.
Mr. LDA
126 Citations2012Ke Zhai, Jordan Boyd‐Graber +2 more
This paper introduces a novel and flexible large scale topic modeling package in MapReduce (Mr. LDA), which uses variational inference, which easily fits into a distributed environment and is easily extensible.
ScientometricsCombining full-text analysis and bibliometric indicators. A pilot study
66 Citations2005Patrick Glenisson, Wolfgang Glänzel +1 more
Full-text analysis and traditional bibliometric methods are combined to improve the efficiency of the individual methods in the mapping of science to prove that such hybrid methodology can be applied to both research evaluation and information retrieval.
Tensor Space Learning for Analyzing Activity Patterns from Video Sequences
59 Citations2007Liang Wang, Christopher Leckie +3 more
A tensor space representation for analyzing human activity patterns in monocular videos that can effectively identify human activities with intra- and inter-person variations on both temporal and spatial scales is described.
ScientometricsOverlaying communities and topics: an analysis on publication networks
47 Citations2011Erjia Yan, Ying Ding +1 more
Results show that a few research topics such as webometrics, bibliometric laws, and language processing, form their own research community; while other research topics contain different research communities, which may be caused by physical distance.
Journal of Engineering and Technology ManagementComparing methods to extract technical content for technological intelligence
45 Citations2013Nils C. Newman, Alan L. Porter +3 more
This presentation compares alternative ways of consolidating messy sets of key terms and compares PCA results to topic modeling results, suggesting good potential to enhance technical intelligence payoffs from database searches on topics of interest.
ScientometricsTen challenges in modeling bibliographic data for bibliometric analysis
40 Citations2012Alfio Ferrara, Silvia Salini
Ten of the most relevant challenges for bibliometric analysis when dealing with multidimensional data are discussed, and a reference data model is proposed that, according to different goals, can help analysis designers and bibliographic experts in working with large collections of bibliographical data.
ScientometricsVisualizing and comparing four facets of scholarly communication: producers, artifacts, concepts, and gatekeepers
38 Citations2012Chaoqun Ni, Cassidy R. Sugimoto +1 more
Parallelized Variational EM for Latent Dirichlet Allocation: An Experimental Evaluation of Speed and Scalability
33 Citations2007Ramesh Nallapati, William W. Cohen +1 more
This work builds parallel implementations of the variational EM algorithm for LDA in a multiproces- sor architecture as well as a distributed setting and indicates that while both the implementations achieve speed-ups, the distributed version achieves dramatic improvements in both speed and scalability.
Search technology, Inc.
20 Citations1989Ruston M. Hunt
In keeping with the corporate mission, Search Technology has developed several innovative concepts in many areas of human-machine interaction, including adaptive aiding, intelligent tutoring, simulation-oriented computerbased instruction, error-tolerant interfaces, intelligent information management, and intelligent information retrieval.
Technology Analysis and Strategic ManagementAnalysis of technological progression by quantitative measures: a comparison of two technologies
11 Citations2013Arho Suominen
The results from these two technologies help to validate measures that would focus, not on specific research organisations or research topics, but on the maturity of technologies, and would be objective, reproducible and valid across different technologies.