A probability distribution model for information retrieval
Information Processing & ManagementPublished 1 January 1989
S. K. M. Wong, Yiyu Yao
Citations44
SJR quartileQ1
SJR score2.06
SNIP2.91
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A probability distribution model for information retrieval is proposed that not only enhances retrieval effectiveness as demonstrated by experiments, but also provides valuable insight into many fundamental concepts introduced over the years in a variety of retrieval models.
Abstract
A probability distribution model for information retrieval is proposed in this article. In contrast to other approaches, each document in the new model is characterized by a simple probability measure on the set of index terms. Based on this interpretation of document representation, two different retrieval strategies are discussed. One is based on utility theory and the other is derived from information theory. The new model not only enhances retrieval effectiveness as demonstrated by experiments, but also provides valuable insight into many fundamental concepts introduced over the years in a variety of retrieval models.
Keywords
Computer Science
IEEE Transactions on Information TheoryApproximating discrete probability distributions with dependence trees
2,640 Citations1968Chee Lap Chow, C. Liu
It is shown that the procedure derived in this paper yields an approximation of a minimum difference in information when applied to empirical observations from an unknown distribution of tree dependence, and the procedure is the maximum-likelihood estimate of the distribution.
Operational Research Quarterly (1970-1977)Utility Theory for Decision Making
2,502 Citations1971Roger Adelson, Peter C. Fishburn
Journal of the American Society for Information ScienceRelevance weighting of search terms
2,068 Citations1976Stephen Robertson, Karen Spärck Jones
This paper examines statistical techniques for exploiting relevance information to weight search terms using information about the distribution of index terms in documents in general and shows that specific weighted search methods are implied by a general probabilistic theory of retrieval.
Theoretical Population BiologyDiversity and dissimilarity coefficients: A unified approach
1,709 Citations1982C. Radhakrishna Rao
Three general methods for obtaining measures of diversity within a population and dissimilarity between populations are discussed and a diversity measure in apportionment of diversity between and within populations is discussed.
Journal of the American Society for Information ScienceA critical analysis of vector space model for information retrieval
313 Citations1986Vijay V. Raghavan, S. K. M. Wong
Notations and definitions necessary to identify the concepts and relationships that are important in modelling information retrieval objects and processes in the context of vector spaces are presented.
IEEE Transactions on Pattern Analysis and Machine IntelligenceEntropy and Distance of Random Graphs with Application to Structural Pattern Recognition
294 Citations1985Andrew K. C. Wong, Manlai You
To synthesize an ensemble of attributed graphs into the distribution of a random graph (or a set of distributions), this work proposes a distance measure between random graphs based on the minimum change of entropy before and after their merging.
Fundamenta InformaticaeOn Generalized Vector Space Model in Information Retrieval
149 Citations1985S. K. M. Wong, Wojciech Ziarko
Journal of the ACMPrecision Weighting—An Effective Automatic Indexing Method
146 Citations1976C. Yu, Gerard Salton
The precision weighting procedure described in the present study uses relevance criteria to weight the terms occurring in user queries as a function of the balance between relevant and nonrelevant documents in which these terms occur; this approximates a semantic know-how of term importance.
IEEE Transactions on Information TheoryCross entropy, dissimilarity measures, and characterizations of quadratic entropy
131 Citations1985C. Radhakrishna Rao, Tapan K. Nayak
It is shown that the square root of the Jensen difference (dissimilarity measure) arising out of a quadratic entropy provides a metric on a probability space.
Journal of the American Society for Information ScienceDocument representation in probabilistic models of information retrieval
51 Citations1981W. Bruce Croft
This article describes how retrieval models which use either independence or dependence assumptions can be extended to include document representatives containing term significance weights and indicates that search strategies based on models modified in this way can further improve the effectiveness of document retrieval systems.
Journal of the ACMA Statistical Model for Relevance Feedback in Information Retrieval
40 Citations1976C. Yu, Wo-Shun Luk +1 more
A statistical model is presented for the investigation of a practical method used in relevance feedback and a region in the plane of the parameters is shown to satisfy the sufficient condition for a better query than the original query.
ACM Transactions on Database SystemsExperiments on the determination of the relationships between terms
27 Citations1979Vijay V. Raghavan, C. Yu
The retrieval effectiveness of an automatic method that uses relevance judgments for the determination of positive as well as negative relationships between terms is evaluated and the importance attached to the term relationship components relative to that of term match component is found to have a substantial effect on the retrieval performance.
Adaptive linear information retrieval models
22 Citations1987P. Bollmann, S. K. M. Wong
This paper proposes a method and gives the precise semantics of the retrieval operations in a system where imprecision is allowed and suggests a way to handle the uncertainty introduced by imprecise data values.
On extending the vector space model for Boolean query processing
21 Citations1986S. K. M. Wong, Wojciech Ziarko +2 more
An information retrieval model, named the Generalized Vector Space Model (GVSM), is extended to handle situations where queries are specified as Boolean expressions and it is shown that this unified model has the advantage of incorporating term correlations into the retrieval process.
Journal of the ACMA Formal Construction of Term Classes
19 Citations1975Clement Yu
While the process is proved to be difficult computatlonally, heumstic methods are applied and results are obtained to illustrate the maximum possible improvement in system performance of retrieval using the formal construction over simple term retrieval.
A statistical similarity measure
14 Citations1987S. K. M. Wong, Yiyu Yao
Within the framework of the vector space models, a statistical similarity measure between document and query is proposed, which provides a natural and consistent interpretation of term occurrence frequencies obtained from autoindexing.