Iterative Information Retrieval Using Fast Clustering and Usage-Specific Genres

Published 1 January 1999

Jussi Karlgren, Ivan Bretan, Johan Dewe, Anders Hallberg, Niklas Wolkert

Citations27

TL;DR

Abstract

This paper describes how collection specific empirically defined stylistics based genre prediction can be brought together together with rapid topical clustering to build an interactive information retrieval interface with multi-dimensional presentation of search results. The prototype presented addresses two specific problems of information retrieval: how to enrich the information seeking dialog by encouraging and supporting iterative refinement of queries, and how to enrich the document representation past the shallow semantics allowed by term frequencies. Searching For More Than Words Today&apos;s tools for searching information in a document database are based on term occurrence in texts. The searcher enters a number of terms and a number of documents where those terms or closely related terms appear comparatively frequently are retrieved and presented by the system in list form. This method works well up to a point. It is intuitively understandable, and for competent users and well e...

Keywords

Computer Science

C4.5: Programs for Machine Learning

23,665 Citations1992J. R. Quinlan

A complete guide to the C4.5 system as implemented in C for the UNIX environment, which starts from simple core learning methods and shows how they can be elaborated and extended to deal with typical problems such as missing data and over hitting.

Information Processing & ManagementTerm-weighting approaches in automatic text retrieval

9,532 Citations1988Gerard Salton, Chris Buckley

This paper summarizes the insights gained in automatic term weighting, and provides baseline single term indexing models with which other more elaborate content analysis procedures can be compared.

Programs for Machine Learning

5,794 Citations1994Steven L. Salzberg, Alberto M. Segre

In his new book, C4.5: Programs for Machine Learning, Quinlan has put together a definitive, much needed description of his complete system, including the latest developments, which will be a welcome addition to the library of many researchers and students.

General Pharmacology The Vascular SystemThe visual display of quantitative information

5,120 Citations1985

ACM SIGIR ForumScatter/Gather

1,425 Citations2017Douglass R. Cutting, David R. Karger +2 more

Reexamining the cluster hypothesis

828 Citations1996Marti A. Hearst, Jan Pedersen

This work systematically evaluates Scatter/Gather in this context and finds significant improvements over similarity search ranking alone and provides evidence validating the cluster hypothesis which states that relevant documents tend to be more similar to each other than to non-relevant documents.

View PDF

ScienceThe Characteristic Curves of Composition

396 Citations1887T. C. Mendenhall

Recognizing text genres with simple metrics using discriminant analysis

293 Citations1994Jussi Karlgren, Douglass R. Cutting

A simple method for categorizing texts into pre-determined text genre categories using the statistical standard technique of discriminant analysis is demonstrated with application to the Brown corpus.

View PDF

Munich Personal RePEc Archive (Ludwig Maximilian University of Munich)Natural language information retrieval: TREC-5 report

82 Citations1996Tomek Strzalkowski, Louise Guthrie +7 more

This paper reports on the joint GE/Lockheed Martin/Rutgers/NYU natural language information retrieval project as related to the 5th Text Retrieval Conference (TREC-5), which uses natural language processing techniques to enhance the effectiveness of full-text document retrieval.

DSpace repository (University of Tartu)Assembling a Balanced Corpus from the Internet

32 Citations1998Johan Dewe, Jussi Karlgren +1 more

The methodology for determining which genres to include is given, which is a double edged problem, involving both vaguely expressed user expectations and establishing categories using large numbers of features which taken singly have low predictive and explanatory power.

View PDF