Combining Heterogeneous Models for Measuring Relational Similarity
Published 27 May 2013
Alisa Zhila, Wen-tau Yih, Christopher Meek, Geoffrey Zweig, Tomáš Mikolov
Citations57
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The overall system consists of two novel general-purpose relational similarity models and three specific word relation models that outperforms the previous best system substantially and achieves a 54.1% relative increase in Spearman's rank correlation.
Abstract
In this work, we study the problem of measuring relational similarity between two word pairs (e.g., silverware:fork and clothing:shirt). Due to the large number of possible relations, we argue that it is important to combine multiple models based on heterogeneous information sources. Our overall system consists of two novel general-purpose relational similarity models and three specific word relation models. When evaluated in the setting of a recently proposed SemEval-2012 task, our approach outperforms the previous best system substantially, achieving a 54.1 % relative increase in Spearman’s rank correlation. 1
Keywords
Computer Science
The Annals of StatisticsGreedy function approximation: A gradient boosting machine.
28,973 Citations2001Jerome H. Friedman
A general gradient descent boosting paradigm is developed for additive expansions based on any fitting criterion, and specific algorithms are presented for least-squares, least absolute deviation, and Huber-M loss functions for regression, and multiclass logistic likelihood for classification.
Recurrent neural network based language model
5,349 Citations2010Tomáš Mikolov, Martin Karafiát +3 more
Results indicate that it is possible to obtain around 50% reduction of perplexity by using mixture of several RNN LMs, compared to a state of the art backoff language model.
Discourse ProcessesAn introduction to latent semantic analysis
4,817 Citations1998Thomas K. Landauer, Peter W. Foltz +1 more
The adequacy of LSA's reflection of human knowledge has been established in a variety of ways, for example, its scores overlap those of humans on standard vocabulary and subject matter tests; it mimics human word sorting and category judgments; it simulates word‐word and passage‐word lexical priming data.
Class-based n -gram models of natural language
2,893 Citations1992Peter F. Brown, P.V. deSouza +3 more
This work addresses the problem of predicting a word from previous words in a sample of text and discusses n-gram models based on classes of words, finding that these models are able to extract classes that have the flavor of either syntactically based groupings or semanticallybased groupings, depending on the nature of the underlying statistics.
Linguistic Regularities in Continuous Space Word Representations
2,882 Citations2013Tomáš Mikolov, Wen-tau Yih +1 more
The vector-space word representations that are implicitly learned by the input-layer weights are found to be surprisingly good at capturing syntactic and semantic regularities in language, and that each relationship is characterized by a relation-specific vector offset.
Proceedings of the AAAI Conference on Artificial IntelligenceToward an Architecture for Never-Ending Language Learning
1,991 Citations2010J. Andrew Carlson, Justin Betteridge +4 more
Word Representations: A Simple and General Method for Semi-Supervised Learning
1,944 Citations2010Joseph Turian, Lev-Arie Ratinov +1 more
This work evaluates Brown clusters, Collobert and Weston (2008) embeddings, and HLBL (Mnih & Hinton, 2009) embeds of words on both NER and chunking, and finds that each of the three word representations improves the accuracy of these baselines.
Applied Physics Letters10.1162/153244303322533223
1,771 Citations2000
This work proposes to fight the curse of dimensionality by learning a distributed representation for words which allows each training sentence to inform the model about an exponential number of semantically neighboring sentences.
Computational LinguisticsEvaluating WordNet-based Measures of Lexical Semantic Relatedness
1,414 Citations2006Alexander Budanitsky, Graeme Hirst
An information-content-based measure proposed by Jiang and Conrath is found superior to those proposed by Hirst and St-Onge, Leacock and Chodorow, Lin, and Resnik, and why distributional similarity is not an adequate proxy for lexical semantic relatedness.
A study on similarity and relatedness using distributional and WordNet-based approaches
880 Citations2009Eneko Agirre, Enrique Alfonseca +4 more
This paper presents and compares WordNet-based and distributional similarity approaches, and pioneer cross-lingual similarity, showing that the methods are easily adapted for a cross-lingsual task with minor losses.
Probase
776 Citations2012Wentao Wu, Hongsong Li +2 more
This paper presents a universal, probabilistic taxonomy that is more comprehensive than any existing ones, and contains 2.7 million concepts harnessed automatically from a corpus of 1.68 billion web pages.
Scalable training of<i>L</i><sup>1</sup>-regularized log-linear models
541 Citations2007Galen Andrew, Jianfeng Gao
This work presents an algorithm Orthant-Wise Limited-memory Quasi-Newton (OWL-QN), based on L-BFGS, that can efficiently optimize the L1-regularized log-likelihood of log-linear models with millions of parameters.
Strategies for training large scale neural network language models
512 Citations2011Tomáš Mikolov, Anoop Deoras +3 more
This work describes how to effectively train neural network based language models on large data sets and introduces hash-based implementation of a maximum entropy model, that can be trained as a part of the neural network model.
A word at a time
413 Citations2011Kira Radinsky, Eugene Agichtein +2 more
This paper proposes a new semantic relatedness model, Temporal Semantic Analysis (TSA), which captures this temporal information in word semantics as a vector of concepts over a corpus of temporally-ordered documents.
Multi-Prototype Vector-Space Models of Word Meaning
355 Citations2010Joseph Reisinger, Raymond J. Mooney
Experimental comparisons to human judgements of semantic similarity for both isolated words as well as words in sentential contexts demonstrate the superiority of this approach over both prototype and exemplar based vector-space models.
National Research Council Canada (Government of Canada)SemEval-2012 Task 2 : measuring degrees of relational similarity
191 Citations2012David Jurgens, Saif M. Mohammad +2 more
A new SemEval task based on identifying the degree of prototypicality for instances within a given class is presented, and the first dataset of graded relational similarity ratings across 79 relation categories is assembled.
A uniform approach to analogies, synonyms, antonyms, and associations
179 Citations2008Peter D. Turney
A supervised corpus-based machine learning algorithm is introduced for classifying analogous word pairs and it is shown that it can solve multiple-choice SAT analogy questions, TOEFL synonyms questions, ESL synonym-antonym questions, and similar-associated-both questions from cognitive psychology.
Machine LearningCorpus-based Learning of Analogies and Semantic Relations
178 Citations2005Peter D. Turney, Michael L. Littman
An algorithm for learning from unlabeled text that can solve verbal analogy questions of the kind found in the SAT college entrance exam and is state-of-the-art for both verbal analogies and noun-modifier relations is presented.
Empirical Methods in Natural Language ProcessingClassifying the Semantic Relations in Noun Compounds via a Domain-Specific Lexical Hierarchy
149 Citations2001Barbara Rosario, Marti A. Hearst
It is concluded that a subset of patients with brain metastases benefit from the addition of checkpoint inhibitors to standard of care therapeutic modalities, including radiotherapy and surgery.
Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign)The semantic interpretation of compound nominals
101 Citations1980Tim Finin
This thesis is an investigation of how a computer can be programmed to understand the class of linguistic phenomena loosely referred to as nominal compounds, i.e. sequences of two or more nouns related through modification.
Algorithm for automatic interpretation of noun sequences
82 Citations1994Lucy Vanderwende
An algorithm for automatically interpreting noun sequences in unrestricted text using broadcoverage semantic information which has been acquired automatically by analyzing the definitions in an on-line dictionary is described.
Polarity Inducing Latent Semantic Analysis
69 Citations2012Wen-tau Yih, Geoffrey Zweig +1 more
The key contribution of this work is to show how to assign signs to the entries in the co-occurrence matrix on which LSA operates, so as to induce a subspace with the desired property.
North American Chapter of the Association for Computational LinguisticsMeasuring Word Relatedness Using Heterogeneous Vector Space Models
44 Citations2012Wen-tau Yih, Vahed Qazvinian
This model consists of a committee of vector space models built on a text corpus, Web search results and theauruses, and measures the semantic word relatedness using the averaged cosine similarity scores.
Joint Conference on Lexical and Computational SemanticsUTD: Determining Relational Similarity Using Lexical Patterns
18 Citations2012Bryan Rink, Sanda M. Harabagiu
A class of NAD-dependent active-site SARM1 inhibitors that function by intercepting NAD hydrolysis and undergoing covalent conjugation with the reaction product adenosine diphosphate ribose are described, validating this mode of inhibition as a viable therapeutic strategy.
Joint Conference on Lexical and Computational SemanticsDuluth : Measuring Degrees of Relational Similarity with the Gloss Vector Measure of Semantic Relatedness
11 Citations2012Ted Pedersen
This paper describes the Duluth systems that participated in Task 2 of SemEval-2012 and relied on variations of the Gloss Vector measure found in the freely available software package WordNet:: Similarity, which was moderately successful for the Class-Inclusion, Similar, Contrast, and Non-Attribute categories of semantic relations.
BUAP: A First Approximation to Relational Similarity Measuring
5 Citations2012Mireya Tovar Vidal, José A. Reyes +4 more
A system proposed for measuring the degree of relational similarity beetwen a pair of words at the Task #2 of Semeval 2012 is described, based on a vectorial representation based on knowledge extracted from WordNet.