Measures of distributional similarity
Published 1 January 1999Open access
Lillian Lee
Citations586
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This work presents an empirical comparison of a broad range of measures; a classification of similarity functions based on the information that they incorporate; and the introduction of a novel function that is superior at evaluating potential proxy distributions.
Abstract
We study distributional similarity measures for the purpose of improving probability estimation for unseen cooccurrences. Our contributions are three-fold: an empirical comparison of a broad range of measures; a classification of similarity functions based on the information that they incorporate; and the introduction of a novel function that is superior at evaluating potential proxy distributions. 1
Keywords
Computer Science
Elements of Information Theory
37,533 Citations2001Thomas M. Cover, Joy A. Thomas
Choice Reviews OnlineFinding groups in data: an introduction to cluster analysis
10,614 Citations1991
IEEE Transactions on Information TheoryDivergence measures based on the Shannon entropy
5,027 Citations1991Jinfeng Lin
A novel class of information-theoretic divergence measures based on the Shannon entropy is introduced, which do not require the condition of absolute continuity to be satisfied by the probability distributions involved and are established in terms of bounds.
Word association norms, mutual information, and lexicography
3,674 Citations1990Kenneth Church, Patrick Hanks
An Information-Theoretic Definition of Similarity
3,662 Citations1998Dekang Lin
This work presents an informationtheoretic definition of similarity that is applicable as long as there is a probabilistic model and demonstrates how this definition can be used to measure the similarity in a number of different domains.
Birkhäuser Basel eBooksElements of Information Theory
2,908 Citations1992Konrad Jacobs
For information theory, messages are elements of a set, and when they are handled, their sole concern is to keep distinct messages distinct, i.e. to make the mappings resulting from the authors' manipulations injective (= one-to-one).
arXiv (Cornell University)Using Information Content to Evaluate Semantic Similarity in a Taxonomy
2,147 Citations1995Philip Resnik
arXiv (Cornell University)Using Information Content to Evaluate Semantic Similarity in a Taxonomy
1,760 Citations1995Philip Resnik
This paper presents a new measure of semantic similarity in an IS-A taxonomy, based on the notion of information content, which performs encouragingly well and is significantly better than the traditional edge counting approach.
IEEE Transactions on Acoustics Speech and Signal ProcessingEstimation of probabilities from sparse data for the language model component of a speech recognizer
1,645 Citations1987Slava M. Katz
The model offers, via a nonlinear recursive procedure, a computation and space efficient solution to the problem of estimating probabilities from sparse data, and compares favorably to other proposed methods.
Automatic retrieval and clustering of similar words
1,555 Citations1998Dekang Lin
A word similarity measure based on the distributional pattern of words allows the automatically constructed thesaurus to be significantly closer to WordNet than Roget Thesaurus is.
Communications of the ACMToward memory-based reasoning
1,232 Citations1986Craig Stanfill, David L. Waltz
The intensive use of memory to recall specific episodes from the past—rather than rules—should be the foundation of machine reasoning.
A stochastic parts program and noun phrase parser for unrestricted text
974 Citations1988Kenneth Church
A program that tags each word in an input sentence with the most likely part of speech has been written and performance is encouraging; a 400-word sample is presented and is judged to be 99.5% correct.
Proceedings of the 17th international conference on Computational linguistics -Automatic retrieval and clustering of similar words
550 Citations1998Dekang Lin
Noun classification from predicate-argument structures
543 Citations1990Donald Hindle
The resulting quasi-semantic classification of nouns demonstrates the plausibility of the distributional hypothesis, and has potential application to a variety of tasks, including automatic indexing, resolving nominal compounds, and determining the scope of modification.
Columbia Academic Commons (Columbia University)Translating Collocations for Bilingual Lexicons: A Statistical Approach
497 Citations1996Frank Smadja, Kathleen McKeown +1 more
A program named Champollion is described which, given a pair of parallel corpora in two different languages and a list of collocations in one of them, automatically produces their translations, to provide a tool for compiling bilingual lexical information above the word level in multiple languages, for different domains.
Integrating multiple knowledge sources to disambiguate word sense
410 Citations1996Hwee Tou Ng, Hian Beng Lee
This approach integrates a diverse set of knowledge sources to disambiguate word sense, including part of speech of neighboring words, morphological form, the unordered set of surrounding words, local collocations, and verb-object syntactic relation.
Machine LearningSimilarity-Based Models of Word Cooccurrence Probabilities
346 Citations1999Ido Dagan, Lillian Lee +1 more
This work proposes a method for estimating the probability of such previously unseen word combinations using available information on “most similar” words, and describes probabilistic word association models based on distributional word similarity, and applies them to two tasks, language modeling and pseudo-word disambiguation.
Text-translation alignment
342 Citations1993Martin Kay, Martin Röscheisen
Diversity: its measurement, decomposition, apportionment and analysis
259 Citations1982C. Radhakrishna Rao
Journal of the American Society for Information SciencePictures of relevance: A geometric analysis of similarity measures
254 Citations1987William Jones, George W. Furnas
A geometric analysis is advanced and its utility demonstrated through its application to six conventional information retrieval similarity measures and a seventh spreading activation measure, intended to complement, and perhaps to guide, the empirical analysis of similarity measures.
Using syntactic dependency as local context to resolve word sense ambiguity
243 Citations1997Dekang Lin
An algorithm is presented that uses the same knowledge sources to disambiguate different words and does not require a sense-tagged corpus and exploits the fact that two different words are likely to have similar meanings if they occur in identical local contexts.
Nonparametric Measures of Association
166 Citations1993Jean Dickinson Gibbons
Information retrieval
141 Citations1965P. J. Gleason
This system was conceived with the user in mind and attempts to demonstrate how each user may contribute to the effectiveness of information searching, to effect new insights into the synthesis of knowledge.
CogPrints (Cogprints)Similarity-based Word Sense Disambiguation
122 Citations1998Yael Karov, Shimon Edelman
Towards the automatic identification of adjectival scales
117 Citations1993Vasileios Hatzivassiloglou, Kathleen McKeown
This paper presents a method to group adjectives according to their meaning, as a first step towards the automatic identification of adjectival scales, and shows how a clustering algorithm can use similarities to produce the groups of adjectives.
A case-based approach to knowledge acquisition for domain-specific sentence analysis
79 Citations1993Claire Cardie
It is concluded that the case-based method provides a promising approach to automated dictionary construction and knowledge acquisition for sentence analysis in limited domains and a novel case retrieval algorithm that uses decision trees to improve the performance of a k-nearest neighbor similarity metric.
ArXiv.orgIntegrating Multiple Knowledge Sources to Disambiguate Word Sense: An Exemplar-Based Approach
78 Citations1996Hwee Tou Ng, Hian Beng Lee
Cooccurrence smoothing for stochastic language modeling
77 Citations1992U. Essen, Volker Steinbiss
Using word-bigram language models, cooccurrence smoothing improved the test-set perplexity by 14% on a German 100000-word text corpus and by 10% on an English 1-million word corpus.
arXiv (Cornell University)Similarity-Based Approaches to Natural Language Processing
76 Citations1997Lillian Lee
The clustering method, which uses the technique of deterministic annealing, represents (to the authors' knowledge) the first application of soft clustering to problems in natural language processing, and is compared to several such nearest-neighbor approaches on a word sense disambiguation task and finds that as a whole, their performance is far superior to that of standard methods.
Computer Speech & LanguageContextual word similarity and estimation from sparse data
72 Citations1995Ido Dagan, Shaul Marcus +1 more
A method that makes local analogies between each specific unobserved co-occurrence and other co-occurrences that contain similar words is presented, based on the assumption that similar word co-occurrences have similar values of mutual information.
arXiv (Cornell University)Exemplar-Based Word Sense Disambiguation: Some Recent Improvements
67 Citations1997Hwee Tou Ng
Statistical sense disambiguation with relatively small corpora using dictionary definitions
62 Citations1995Alpha Luk
Using the definition-based conceptual co-occurrence data collected from the relatively small Brown corpus, the sense disambiguation system achieves an average accuracy comparable to human performance given the same contextual information.
Memory-based learning
42 Citations1997Jakub Zavrel, Walter Daelemans
It is argued that feature weighting methods in the Memory-Based paradigm can offer the advantage of automatically specifying a suitable domain-specific hierarchy between most specific and most general conditioning information without the need for a large number of parameters.
Isolated word recognition using hidden Markov models
35 Citations2005Kohei Sugawara, Masafumi Nishimura +3 more
This paper investigated smoothing techniques for alleviating the problem due to insufficient amount of training data for isolated word recognition and found that the confusion matrix between a pair of label prototypes was particularly effective for the problem.
Tilburg University Research PortalMemory-Based Learning: Using Similarity for Smoothing
33 Citations1997Jakub Zavrel, Walter Daelemans
Smoothing of automatically generated selectional constraints
29 Citations1993Ralph Grishman, John Sterling
An approach to automatically make suitable generalizations is reported on: using the co-occurrence data to compute a confusion matrix relating individual words, and then using the confusion matrix to smooth the original frequency data.
The MIT Press eBooksDo We Need Linguistics When We Have Statistics? A Comparative Analysis of the Contributions of Linguistic Cues to a Statistical Word Grouping System
18 Citations1996
The results show that linguistic knowledge causes a significant increase in the performance of the system, and it is discussed how these positive restdts can be generalized to other problems in statistical NLP.
Isolated Word Recognition Using Hidden Markov Models
4 Citations1988Stephan Euler, Dietrich E. Wolf
An automatic speaker independent isolated word recognition system based on continuous hidden Markov models is presented using multidimensional spherically invariant density functions which describe the statistical properties of words.