Topic Detection by Clustering Keywords
Published 1 September 2008
Christian Wartena, Rogier Brussee
Citations151
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
Evaluation on Wikipedia articles shows that clusters of keywords correlate strongly with the Wikipedia categories of the articles, and a newly proposed term distribution taking co-occurrence of terms into account gives best results.
Abstract
We consider topic detection without any prior knowledge of category structure or possible categories. Keywords are extracted and clustered based on different similarity measures using the induced k-bisecting clustering algorithm. Evaluation on Wikipedia articles shows that clusters of keywords correlate strongly with the Wikipedia categories of the articles. In addition, we find that a distance measure based on the Jensen-Shannon divergence of probability distributions outperforms the cosine similarity. In particular, a newly proposed term distribution taking co-occurrence of terms into account gives best results.
Keywords
Computer SciencePhysics and Astronomy
Foundations of statistical natural language processing
9,996 Citations1999Christopher D. Manning, Hinrich Schütze
Discourse ProcessesAn introduction to latent semantic analysis
4,817 Citations1998Thomas K. Landauer, Peter W. Foltz +1 more
The adequacy of LSA's reflection of human knowledge has been established in a variety of ways, for example, its scores overlap those of humans on standard vocabulary and subject matter tests; it mimics human word sorting and category judgments; it simulates word‐word and passage‐word lexical priming data.
University of Minnesota Digital Conservancy (University of Minnesota)A Comparison of Document Clustering Techniques
2,458 Citations2000Michael Steinbach, George Karypis +1 more
This paper compares the two main approaches to document clustering, agglomerative hierarchical clustering and K-means, and indicates that the bisecting K-MEans technique is better than the standard K-Means approach and as good or better as the hierarchical approaches that were tested for a variety of cluster evaluation metrics.
Machine LearningUnsupervised Learning by Probabilistic Latent Semantic Analysis
2,443 Citations2001Thomas Hofmann
This paper proposes to make use of a temperature controlled version of the Expectation Maximization algorithm for model fitting, which has shown excellent performance in practice, and results in a more principled approach with a solid foundation in statistical inference.
Fast and effective text mining using linear-time document clustering
862 Citations1999Bjornar Larsen, Chinatsu Aone
An unsupervised, near-linear time text clustering system that offers a number of algorithm choices for each phase, and a refinement to center adjustment, “vector average damping,” that further improves cluster quality.
Jensen-Shannon divergence and Hilbert space embedding
621 Citations2004Bent Fuglede, Flemming Topsøe
The Jensen-Shannon divergence (JSD) and Hilbert space embedding is described and the set of distributions with the metric /spl radic/JSD can be embedded isometrically into Hilbert space and the embedding can be identified.
Using latent semantic analysis to improve access to textual information
607 Citations1988Susan Dumais, George W. Furnas +3 more
Initial tests find this completely automatic method widely applicable and a promising way to improve users' access to many kinds of textual materials, or to objects and services for which textual descriptions are available.
Lecture notes in computer scienceEmergent Semantics from Folksonomies: A Quantitative Study
62 Citations2006Lei Zhang, Xian Wu +1 more
This paper investigates using a bottom-up approach for semantically annotating web resources as supported by the now widely popular social bookmarks services on the web where users can annotate and categorize web resources using “tags” freely choosen by the user without any pre-existing global semantic model.
Information Processing & ManagementTopic analysis using a finite mixture model
50 Citations2003Hang Li, Kenji Yamanishi
Lecture notes in computer scienceAnalysis of Clustering Algorithms for Web-Based Search
29 Citations2002Sven Meyer zu Eissen, Benno Stein
This paper presents results of a comprehensive analysis of clustering algorithms in connection with document categorization, relating to exemplar-based, hierarchical, and density-based clusteringgorithms.
Information Processing & ManagementText classification using ESC-based stochastic decision lists
29 Citations2002Hang Li, Kenji Yamanishi
Lecture notes in computer scienceA Hierarchical Document Clustering Environment Based on the Induced Bisecting k-Means
27 Citations2006Francesco Archetti, P. Campanelli +2 more
This paper presents a Knowledge Discovery System for document processing and clustering, and the clustering algorithm implemented in this system, called Induced Bisecting k-Means, outperforms the Standard Bisected k- means and is particularly suitable for on line applications when computational efficiency is a crucial aspect.