Unsupervised and supervised clustering for topic tracking
Published 1 September 2001
Martin Franz, Todd Ward, Jason S. McCarley, Wei-Jing Zhu
Citations81
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
Important differences between two styles of document clustering in the context of Topic Detection and Tracking are investigated in both the design and the evaluation of TDT systems.
Abstract
We investigate important differences between two styles of document clustering in the context of Topic Detection and Tracking. Converting a Topic Detection system into a Topic Tracking system exposes fundamental differences between these two tasks that are important to consider in both the design and the evaluation of TDT systems. We also identify features that can be used in systems for both tasks.
Keywords
Computer Science
Program electronic library and information systemsAn algorithm for suffix stripping
8,136 Citations1980Martin Porter
An algorithm for suffix stripping is described, which has been implemented as a short, fast program in BCPL, and performs slightly better than a much more elaborate system with which it has been compared.
Text REtrieval ConferenceOkapi at TREC
2,229 Citations1994Stephen Robertson, Steve Walker +3 more
Much of the work involved investigating plausible methods of applying Okapi-style weighting to phrases, and expansion using terms from the top documents retrieved by a pilot search on topic terms was used.
The DET curve in assessment of detection task performance
1,401 Citations1997Alvin F. Martín, George R. Doddington +3 more
The DET Curve is introduced as a means of representing performance on detection tasks that involve a tradeoff of error types and why it is likely to produce approximately linear curves.
Research Showcase @ Carnegie Mellon University (Carnegie Mellon University)Topic Detection and Tracking Pilot Study Final Report
951 Citations2018James Allan, Jaime Carbonell +3 more
A study of retrospective and on-line event detection
731 Citations1998Yiming Yang, Tom Pierce +1 more
This paper applied hierarchical and non-hierarchical document clustering algorithms to a corpus of 15,836 stories, focusing on the exploitation of both content and temporal information, and found the resulting cluster hierarchies highly informative for retrospective detection of previously unidentified events.
ACM SIGIR ForumOn-Line New Event Detection and Tracking
653 Citations2017James Allan, Ron Papka +1 more
On-line new event detection and tracking
524 Citations1998James Allan, Ron Papka +1 more
The approach to detection uses a single pass clustering algo-rithm and a novel thresholding model that incorporates the properties of events as a major component and the value of \surprising" features that have unusual occurrence characteristics are discussed.
IEEE Intelligent Systems and their ApplicationsLearning approaches for detecting and tracking news events
338 Citations1999Yi Yang, Jaime Carbonell +4 more
The authors extend existing supervised-learning and unsupervised-clustering algorithms to allow document classification based on the information content and temporal aspects of news events to be classified using manually segmented documents.
The TREC Spoken Document Retrieval Track: A Success Story.
200 Citations1999John S. Garofolo, Cedric G. P. Auzanne +1 more
The SDR Track can be declared a success in that it has provided objective, demonstrable proof that this technology can be successfully applied to realistic audio collections using a combination of existing technologies and that it can be objectively evaluated.
Improving text categorization methods for event tracking
156 Citations2000Yiming Yang, Tom Ault +2 more
This work adapted several supervised text categorization methods, specifically several new variants of the k-Nearest Neighbor (kNN) algorithm and a Rocchio approach, to track events, and significantly reduced the variance in performance of the event tracking system over different data collections, suggesting a robust solution for parameter optimization.
An investigation of linguistic features and clustering algorithms for topical document clustering
130 Citations2000Vasileios Hatzivassiloglou, Luis Gravano +1 more
A statistical model for combining similarity information from multiple sources is described and applied to DARPA's Topic Detection and Tracking phase 2 (TDT2) data, alleviating the need for extensive search in order to determine optimal weights for combining input features.
Topic Detection and Tracking using idf-Weighted Cosine Coefficient
97 Citations1999Johannes Schultz
This research achieved very competitive results for tracking using a very simple method of feature selection, without word stemming and without a score normalization scheme, based on the well-known idf -weighted cosine coefficient similarity metric.
NIST's 1998 topic detection and tracking evaluation (TDT2)
70 Citations1999Jon Fiscus, George R. Doddington +2 more
The 1998 TDT evaluation (TDT2) continues the work of the TDT pilot study conducted in 1997 and is the first open evaluation of TDT tasks.
Topic detection in broadcast news
62 Citations1999Frederick Walls, Hubert Jin +2 more
A system for the Topic Detection and Tracking (TDT) detection task concerned with the unsupervised grouping of news stories according to topic uses an incremental k-means algorithm for clustering stories and utilizes a probabilistic document similarity metric and a traditional vector-space metric for comparing stories.
The TREC-8 Filtering Track Final Report.
46 Citations1999David A. Hull, Stephen Robertson
Story Segmentation and Topic Detection in the Broadcast News Domain
18 Citations1999S. Dharanipragada, Martin Franz +3 more
Algorithms for story segmentation and topic detection are presented that use a decision tree based probabilistic model and an incremental clustering algorithm that employs a novel dynamic clusterdependent similarity measure between documents and clusters.
Topic Tracking with the PRISE Information Retrieval System
9 Citations2009Douglas W. Oard
The results indicate that the simple approach produced a credible system, but comparison with results achieved by other systems indicates that there is room for improvement.