The Cluster Hypothesis Revisited

ACM SIGIR ForumPublished 2 August 2017

Ellen M. Vdorhees

Citations174

TL;DR

While the absolute performance of a search on a particular collection is dependent on the palrwise similarity of the relevant documents, the relative effectiveness of clustered retrieval versus sequential retrieval is independent of this factor.

Abstract

A new means of evaluating the cluster hypothesis is introduced and the results of such an evaluation are presented for four collections. The results of retrieval experiments comparing a sequential search, a cluster-based search, and a search of the clustered collection in which individual documents are scored against the query are also presented. These results indicate that while the absolute performance of a search on a particular collection is dependent on the palrwise similarity of the relevant documents, the relative effectiveness of clustered retrieval versus sequential retrieval is independent of this factor. However, retrieval of entire clusters in response to a query usually results in a poorer performance than retrieval of individual documents from clusters.

Keywords

Computer Science

Communications of the ACMExtended Boolean information retrieval

1,038 Citations1983Gerard Salton, Edward A. Fox +1 more

A new, extended Boolean information retrieval system is introduced which is intermediate between the Boolean system of query processing and the vector processing model, and Laboratory tests indicate that the extended system produces better retrieval output than either the Boolean or thevector processing systems.

View PDF

Information Storage and RetrievalThe use of hierarchic clustering in information retrieval

522 Citations1971N. Jardine, C. J. van Rijsbergen

It is shown that cluster-based retrieval strategies can be devised which are as effective as linear associative retrieval strategies and much more efficient.

Information SystemsA model of cluster searching based on classification

143 Citations1980W. Bruce Croft

A probabilistic model of cluster searching based on query classification is described and it is tested with retrieval experiments which indicate that it can be more effective than heuristic cluster searches and cluster searches based on other models.

Journal of DocumentationHIERARCHIC AGGLOMERATIVE CLUSTERING METHODS FOR AUTOMATIC DOCUMENT CLASSIFICATION

117 Citations1984Alan Griffiths, Lesley A. Robinson +1 more

The results would suggest that the single linkage method, which has been used extensively in previous work on document clustering, is not the most effective procedure of those tested, although it should be emphasized that the experiments have used only small document test collections.

Extending the boolean and vector space models of information retrieval with p-norm queries and multiple concept types

107 Citations1983Edward A. Fox

Four experimental test collections are employed to prove that interpreting Boolean queries with p-norm techniques leads to substantial improvements in retrieval effectiveness, as shown by preliminary experimental studies with collections in computer and information science.

Information Processing & ManagementDocument clustering: An evaluation of some experiments with the cranfield 1400 collection

88 Citations1975C. J. van Rijsbergen, W. Bruce Croft

A variety of retrieval strategies applied to this hierarchy are evaluated in terms of effectiveness and efficiency and comparisons are made between these results and those of similar experiments in document clustering on the Smart project.

Journal of DocumentationA TEST FOR THE SEPARATION OF RELEVANT AND NON‐RELEVANT DOCUMENTS IN EXPERIMENTAL RETRIEVAL COLLECTIONS

59 Citations1973C. J. van Rijsbergen, Karen Spärck Jones

A procedure for characterizing this separation of relevant and non‐relevant documents for a collection is presented, which can be used to show whether proposed modifications of the base system are likely to be useful.

Information Storage and RetrievalFurther experiments with hierarchic clustering in document retrieval

38 Citations1974C. J. van Rijsbergen

A framework for the evaluation of cluster-based retrieval strategies is constructed and these strategies are shown to be dependent on the method of cluster representation (cluster profile) adopted.