An approach to the automatic construction of global thesauri
Information Processing & ManagementPublished 1 January 1990
Carolyn J. Crouch
Citations140
SJR quartileQ1
SJR score2.06
SNIP2.91
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
An approach to the automatic generation of global thesauri based on the term discrimination value model of Salton, Yang, and Yu and on an appropriate clustering algorithm is described.
Abstract
The benefits of a well constructed thesaurus to an information retrieval system have long been recognized by both researchers and practitioners in the field. Previous experiments have investigated the construction of thesauri by manual, semiautomatic, and automatic means. Automatic thesaurus generation in particular has proven to be an especially difficult problem. This paper examines both early and current approaches to automatic thesaurus construction and describes an approach to the automatic generation of global thesauri based on the term discrimination value model of Salton, Yang, and Yu and on an appropriate clustering algorithm. This method has been implemented and applied to two document collections. Preliminary results indicate that this method, which produces improvements in retrieval performance in excess of 10 and 15 percent in the test collections, is viable and worthy of continued investigation.
Keywords
Computer Science
Information Processing & ManagementTerm-weighting approaches in automatic text retrieval
9,532 Citations1988Gerard Salton, Chris Buckley
This paper summarizes the insights gained in automatic term weighting, and provides baseline single term indexing models with which other more elaborate content analysis procedures can be compared.
Communications of the ACMA vector space model for automatic indexing
7,434 Citations1975Gerard Salton, Anita M.-Y. Wong +1 more
An approach based on space density computations is used to choose an optimum indexing vocabulary for a collection of documents, demonstating the usefulness of the model.
The Computer JournalA General Theory of Classificatory Sorting Strategies: 1. Hierarchical Systems
1,684 Citations1967G. N. Lance, W. T. Williams
It is shown that the computational behaviour of a hierarchical sorting-strategy depends on three properties, which are established for five conventional strategies and four measures.
Journal of the American Society for Information ScienceA theory of term importance in automatic text analysis
388 Citations1975Gerard Salton, Chul‐Su Yang +1 more
Most existing automatic content analysis and indexing techniques are based on word frequency characteristics applied largely in an ad hoc manner, but terms exhibiting high occurence frequencies in individual documents are often useful for high recall performance, whereas terms with low frequency in the whole collection are useful forhigh precision.
Journal of the American Society for Information ScienceI3R: A new approach to the design of document retrieval systems
334 Citations1987W. Bruce Croft, Roger H. Thompson
A system that provides a number of FACILITIES and SEARCH STRATEGIES based on an EMPHASIS on domain knowledge used for refining the model of the information need, and the provision of a blowing mechanism that allows the user to NAVIGATE through the knowledge base.
Information Processing & ManagementImplementing agglomerative hierarchic clustering algorithms for use in document retrieval
323 Citations1986Ellen M. Voorhees
The implementations of three agglomerative hierarchic clustering algorithms that exploit sparsity so that collections much larger than the algorithms'' worst case running times would suggest can be clustered.
Journal of the American Society for Information ScienceA critical analysis of vector space model for information retrieval
313 Citations1986Vijay V. Raghavan, S. K. M. Wong
Notations and definitions necessary to identify the concepts and relationships that are important in modelling information retrieval objects and processes in the context of vector spaces are presented.
The Computer JournalThe Construction of Hierarchic and Non-Hierarchic Classifications
251 Citations1968N. Jardine, Robin Sibson
A theoretical framework within which the properties of cluster methods, which operate on data in the form of a dissimilarity coefficient on a set of objects, may be discussed is outlined.
Journal of the ACMLocal Feedback in Full-Text Retrieval Systems
234 Citations1977Rony Attar, Aviezri S. Fraenkel
Local clustering is practical also for large databases and appears to improve overall performance, especially if metrical constraints and weighting by proximity are embedded m the local feedback.
Journal of DocumentationINFORMATION RETRIEVAL THROUGH MAN‐MACHINE DIALOGUE
234 Citations1977Robert N Oddy
Initial tests with a prototype program indicate that a performance equal to that obtainable from a more conventional on‐line retrieval system is possible without obliging the user to formulate his query.
The Computer JournalThe Retrieval Effects of Query Expansion on a Feedback Document Retrieval System
144 Citations1983Alan F. Smeaton
Information Processing & ManagementThe selection of good search terms
122 Citations1981C. J. van Rijsbergen, David J. Harper +1 more
This paper tackles the problem of how one might select further search terms, using relevance feedback, given the search terms in the query, by generating a number of different spanning trees from a variety of association measures.
eCommons (Cornell University)The Effectiveness and Efficiency of Agglomerative Hierarchic Clustering in Document Retrieval
122 Citations1985Ellen M. Voorhees
The main goal of this thesis is to compare clustered file searches and inverted file searches in order to determine under what circumstances one search is to be preferred over the other.
Journal of the American Society for Information ScienceUsing interdocument similarity information in document retrieval systems
109 Citations1986Alan Griffiths, H. Claire Luckhurst +1 more
A comparison of the use of clusters consisting of a document together with that document with which it is most similar with conventional best match searches suggests that the two types of search are of comparable effectiveness, but they retrieve noticeably different sets of relevant documents.
Extending the boolean and vector space models of information retrieval with p-norm queries and multiple concept types
107 Citations1983Edward A. Fox
Four experimental test collections are employed to prove that interpreting Boolean queries with p-norm techniques leads to substantial improvements in retrieval effectiveness, as shown by preliminary experimental studies with collections in computer and information science.
ACM SIGIR ForumA Study of the Overlap Among Document Representations
103 Citations2017Jeffrey Katzer, Judith A. Tessier +2 more
Results indicate that for any two representations considered, performance values differed slightly while overlap scores were low, thus supporting the evidence that recall and precision as performance measures mask differences between the sets of retrieved documents.
International ACM SIGIR Conference on Research and Development in Information RetrievalVector space model of information retrieval: a reevaluation
89 Citations1984S. K. M. Wong, Vijay V. Raghavan
The methods used in the current vector based systems are in conflict with the premises of the vector space model, and this investigation will lead to a clearer understanding of the issues and problems in using the vectors space model in information retrieval.
The use of cluster hierarchies in hypertext information retrieval
88 Citations1989Donald B. Crouch, Carolyn J. Crouch +1 more
An hierarchical structure is described which effectively supports the graphical traversal of a document collection in a hypertext system and an overview of an interactive browser based on cluster hierarchies is provided.
Journal of the American Society for Information ScienceRelational thesauri in information retrieval
84 Citations1985Yih‐Chen Wang, James Vandendorpe +1 more
While thesauri that combine a number of relations together have been most effective in a batch environment, there is reason to believe that individual relations will be more useful in an interactive retrieval system that presents index terms to the user and allows him to choose those that best convey his meaning.
eCommons (Cornell University)Characterization of Two New Experimental Collections in Computer and Information Science Containing Textual and Bibliographic Concepts
72 Citations1983Edward A. Fox
A cluster-based approach to thesaurus construction
64 Citations1988Carolyn J. Crouch
This paper describes one approach to the automatic generation of global thesauri, based on the discrimination value model of Salton, Yang, and Yu and on an appropriate clustering algorithm, which has been implemented and applied to two document collections.
ACM SIGIR ForumLexical relations
62 Citations1980Edward A. Fox
The SMART type of information retrieval system is described and the applicability of the above-mentioned lexicon to such a system is discussed, and the list of lexical relations included in the ECD is expanded and organized to be more effective for retrieval, partially along the lines suggested by Evens and Smith.
Building a large thesaurus for information retrieval
50 Citations1988Edward A. Fox, J. Terry Nutter +3 more
Thesaurus in the form of a large semantic network to support interactive query expansion and search by end users is built, being built by analyzing and merging data from several large English dictionaries.
Journal of the American Society for Information ScienceNatural language processing in information retrieval
43 Citations1986Tamas E. Doszkocs
Information Processing & ManagementAn algorithm for the calculation of exact term discrimination values
28 Citations1985Peter Willett
Evidence is presented to show that the relationship between term discrimination and term frequency is crucially dependent upon the type of inter-document similarity measure that is used for the calculation of the discrimination values.
Journal of the American Society for Information ScienceTesting of a natural language retrieval system for a full text knowledge base
26 Citations1984Lionel M. Bernstein, Robert E. Williamson
ANNOD is a retrieval system which combines use of probabilistic, linguistic, and empirical means to rank individual paragraphs of full text for their similarity to natural language queries proposed by users.
Information Processing & ManagementAn improved algorithm for the calculation of exact term discrimination values
21 Citations1988Abdelmoula El-Hamdouchi, Peter Willett
An efficient algorithm for the calculation of term discrimination values that may be used when the interdocument similarity measure used is the cosine coefficient and when the document representatives have been weighted using one particular term-weighting scheme is described.
Computers and the HumanitiesOn the role of words and phrases in automatic text analysis
15 Citations1976Gerard Salton, Anita M.-Y. Wong
A model, known as discrimination value analysis is introduced which assigns an appropriate role in the indexing operation to the terms, term phrases, and thesaurus classes.
Information Processing & ManagementAn analysis of approximate versus exact discrimination values
15 Citations1988Carolyn J. Crouch
The experimental results show that the rankings produced by the exact approach and by a centroid-based algorithm suggested by the author are highly compatible, and indicate that a previously suggested method involving the calculation of exact discrimination values cannot be recommended in view of the excessive cost associated with such an approach.
ACM SIGIR ForumIncorporating different search models into one document retrieval system
14 Citations1981W. Bruce Croft
Journal of the American Society for Information ScienceDocument retrieval experiments using cluster analysis
12 Citations1973Jack Minker, Eero Peltola +1 more
This study tends to support the conclusion of Sparck‐Jones that weighted index terms provide better retrieval performance than unweighted terms, and concludes that the results are highly dependent upon the document collection, and the technique should be employed with caution.
ACM SIGIR ForumExpert systems and information retrieval
10 Citations1997Gerard Salton
The conclusion is reached that expert systems are unlikely to provide much relief in ordinary retrieval environments and simpler and more effective retrieval systems can be implemented by falling back on methodologies proposed and evaluated over twenty years ago that operate without expert system intervention.
VTechWorks (Virginia Tech)Building the CODER Lexicon: The Collins English Dictionary and Its Adverb Definitions
10 Citations1986Edward A. Fox, Robert C. Wohlwend +2 more
It is hoped that definitions for adverbs and other words will be parsed so that the relational lexicon being constructed will include many additional relationships and other knowledge about words and their usage.
Information Processing & ManagementThe computation of discrimination values
8 Citations1975Robert G. Crawford
Results have been reported showing the usefulness of discrimination value in automatic construction of dictionaries for information retrieval, and a relatively efficient algorithm is presented and an example is given.
ACM SIGIR ForumExpert systems and information retrieval
7 Citations1987Gerard Salton
The conclusion is reached that expert systems are unlikely to provide much relief in ordinary retrieval environments and simpler and more effective retrieval systems than those currently in use can be implemented by falling back on methodologies proposed and evaluated over twenty years ago.
Information Processing & ManagementLocal feedback and intelligent automatic query expansion
5 Citations1983Pirkko Pietiläinen
An iterative method for information retrieval uses searchonyms found from the previously retrieved set of documents in query expansion to form the feedback seed, which is subsequently used in query reformulation.
The Computer JournalDisc Database Efficiency: A Scheme for Detailed Assessment Based on Semi-Markov Models
4 Citations1983Chi-nam Leung